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ASSESSMENT  OF  ARMY  AVIATORS'  ABILITY  TO  PERFORM 
INDIVIDUAL  AND  COLLECTIVE  TASKS  IN  THE 
AVIATION  NETWORKED  SIMULATOR  (AIRNET) 


Introduction 

Background 

Assessments  by  the  Directorate  of  Combat  Developments 
(1982,  1983,  1986)  at  the  U.S.  Army  Aviation  Center 
(USAAVNC) ,  Fort  Rucker,  Alabama,  revealed  deficiencies  in 
training  Army  aviators  to  perform  collective  tasks.1 
Specifically,  the  Battlefield  Development  Plan  (1986) 
identified  deficiencies  in  the  training  of  air-to-air, 
antiarmor,  air  assault  operations,  suppression  of  enemy  air 
defense,  special  operations  missions,  aerial  reconnaissance, 
combat  maneuvers,  search  and  rescue,  and  target  acquisition 
and  handover.  The  Battlefield  Development  Plan  traced  the 
training  deficiencies  to  constraints  that  prevent  the  conduct 
of  adequate  training  on  collective  tasks  in  the  actual 
equipment.  Among  the  most  important  constraints  identified 
are  the  following: 

•  training  ranges  are  insufficient  in  number,  size,  and 
topograghic  diversity  for  conducting  effective 
collective  task  training; 

•  the  high  cost  of  aircraft,  fuel,  ordinance,  and 
logistic  support  limits  the  frequency  with  which 
collective  task  training  exercises  can  be  conducted; 
and 

•  the  conduct  of  collective  task  training  in  the  actual 
equipment  under  realistic  conditions  increases  the 
likelihood  of  crashes  and  laser  accidents. 

Since  such  constraints  are  difficult  to  overcome.  Army 
officials  reasoned  that  collective  task  training  may  be 
accomplished  more  cost  effectively  in  a  training  device. 

This  reasoning  led  the  USAAVNC  to  establish  a  Memorandum  of 
Understanding  with  the  Defense  Advance  Research  Projects 
Agency  (DARPA)  in  1987  to  develop  a  prototype  combined  arms 
tactical  trainer  that  can  be  used  to  eliminate  some  or  all  of 
the  deficiencies  in  collective  task  training.  The  Memorandum 
of  Understanding  between  USAAVNC  and  DARPA  led,  in  turn,  to 
the  development  of  the  aviation  networked  simulation  system, 
which  is  referred  to  as  AIRNET. 


1As  the  term  is  used  here,  a  collective  task  refers  to  a  task 
for  which  success  depends  on  the  performance  of  (a)  the  crews 
of  two  or  more  aircraft  or  (b)  the  crew  of  one  or  more 
aircraft  and  the  crew  of  one  or  more  ground  units. 
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On  June  16,  1988,  the  Directorate  of  Training  and  Doctrine 
(DOTD)  tasked  the  Field  Unit  at  Fort  Rucker,  Alabama,  to  assist 
in  evaluating  the  effectiveness  of  AIRNET  for  training  Army 
Training  and  Evaluation  Program/Mission  Training  Plan  (ARTEP/MTP) 
and  Aircrew  Training  Manual  (ATM)  tasks.  In  addition,  the  Fort 
Rucker  Field  Unit  was  tasked  to  provide  recommendations  about 
design  modifications  that  are  likely  to  increase  AIRNET's 
training  effectiveness. 

A  valid  and  comprehensive  assessment  of  the  training 
effectiveness  of  any  training  device  requires  the  conduct  of 
transfer  of  training  research  (research  that  measures  the 
extent  to  which  training  in  the  device  transfers  to 
performance  in  the  operational  equipment) .  However,  because 
transfer  of  training  research  is  costly  and  time  consuming, 
it  was  concluded  that  transfer  of  training  research  cannot  be 
justified  until  the  AIRNET  design  has  stabilized.  This 
research  is  based  on  the  premise  that  useful,  albeit  limited, 
information  about  AIRNET's  training  value  can  be  obtained 
from  a  study  of  experienced  crewmembers’  ability  to  perform 
selected  flying  tasks  in  AIRNET.  The  rationale  underlying 
the  decision  to  investigate  in-simulator  performance  is  as 
follows:  if  a  task  that  an  experienced  crewmember  performs 

routinely  in  the  aircraft  cannot  be  performed  adequately  by 
the  crewmember  in  AIRNET,  it  is  highly  unlikely  that 
effective  training  on  that  task  can  be  accomplished  in 
AIRNET.  It  is  important  to  emphasize,  however,  that  positive 
transfer  of  training  cannot  be  inferred  from  evidence  that 
experienced  aviators  can  perform  tasks  adequately  in  AIRNET. 


Objectives 

The  primary  objective  of  the  research  was  to  assess 
experienced  crewmembers'  ability  to  perform  collective  tasks 
in  AIRNET.  However,  because  all  collective  tasks  consist  of 
a  sample  of  individual  tasks,  a  second  objective  of  the 
research  was  to  assess  experienced  crewmembers'  ability  to 
perform  selected  individual  tasks  in  AIRNET.  A  third 
objective  was  to  identify  AIRNET  design  attributes  that  may 
contribute  to  crewmembers'  inability  to  perform  either 
collective  tasks  or  individual  tasks.  A  fourth  objective  was 
to  assess  participants'  opinions  about  AIRNET's  effectiveness 
for  training  specific  flying  tasks. 


Method 

The  research  was  conducted  using  the  AIRNET  system 
located  at  the  USAAVNC,  Fort  Rucker,  Alabama.  The  research 
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required  4  weeks  for  crewmember  instruction  and  data 
collection.  The  following  subsections  describe  the  personnel 
who  participated  in  the  research,  the  AIRNET  system,  the  tasks 
investigated,  the  performance  assessment  methods,  and  the 
evaluation  procedures. 


Research  Participants 


Crewmembers 

Fifteen  members  of  an  operational  attack  helicopter 
company  participated  in  the  research.  The  participating 
crewmembers  included  one  company  commander,  four  AH-64A 
attack  crews  (four  pilots  and  four  copilot  gunners  [CPGs]), 
and  three  OH-58C  scout  crews  (three  pilots  and  three 
aeroscout  observers) .  The  crewmembers'  ages  varied  from  20 
to  45  years;  the  median  age  was  26  years.  The  experience 
level  of  the  crewmembers  ranged  from  8  months  to  over  17 
years  of  active  duty  military  service.  Thirteen  crewmembers 
had  served  in  the  same  operational  unit  and  had  trained 
together  as  crews  and  teams  for  more  than  15  months.  Two 
crewmembers  were  recent  graduates  of  the  Initial  Entry  Rotary 
Wing  ( IERW)  course  and  had  served  in  the  ur.it  for  less  than  6 
months.  All  of  the  aviators  were  mission  qualified  and  had 
ratings  of  Readiness  Level  (RL)  1. 

Most  crewmembers  had  logged  at  least  a  moderate  number 
of  flight  hours  at  the  time  they  participated  in  the  research 
(see  Table  1) .  Although  aeroscout  observers  had  logged 
considerably  fewer  hours  than  the  other  crewmembers,  their 
flight  hours  are  typical  of  the  moderately  experienced 


Table  1 


Flight  Hours  Logged  by  Participating  Crewmembers 


Group 

Flight  Hours  Logged 
Range 

Median 

Attack  Pilots 

200  -  3700 

2512 

Attack  Copilot/Gunner 

210  -  700 

375 

Scout  Pilots 

375  -  670 

410 

Aeroscout  Observer 

180  -  200 

200 
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aeroscout  observer.  As  a  group,  the  attack  pilots  had  logged 
far  more  flight  hours  than  the  other  three  types  of 
crewmembers .  Only  one  attack  pilot  can  be  considered  an 
inexperienced  aviator  (200  flight  hours  logged) .  Although 
the  attack  CPGs  and  the  scout  pilots  reported  fewer  flight 
hours  than  the  attack  pilots,  most  of  them  can  be  considered 
moderately  or  highly  experienced  aviators;  only  one  attack 
CPG  can  be  considered  a  novice  aviator  (210  flight  hours 
logged) . 


Evaluators 

The  evaluation  team  consisted  of  seven  members.  Four 
subject  matter  experts  (SMEs)  served  as  performance  raters 
during  the  research.  All  performance  raters  were  aviators 
(two  attack  and  two  scout)  who  had  experience  in  the 
instruction  of  attack  helicopter  and  air  cavalry/ 
reconnaissance  troop  operations. 

A  representative  from  the  Threat  Division,  Directorate 
of  Combat  Developments,  supervised  and  manipulated  the  semi- 
automated  threat  forces  from  the  SIMNET  system  located  at 
Fort  Knox,  Kentucky.  The  threat  SME  provided  subjective 
assessments  of  the  tactics  that  crewmembers  employed  against 
the  automated  threat  force . 

A  reliability,  availability,  and  maintainability  (RAM) 
engineer  from  DOTD  was  responsible  for  recording  the  type  and 
frequency  of  system  malfunctions.  The  en_ ineer  also  was 
responsible  for  identifying  any  RAM  problems  other  than 
malfunctions  associated  with  AIRNET. 

The  final  member  of  the  evaluation  team,  a  research 
psychologist,  was  responsible  for  coordinating  all  aspects  of 
the  research,  including  data  collection  and  analysis. 


AIRNET  System 

The  USAAVNC  AIRNET  system  and  its  components  are 
described  in  detail  elsewhere  (see  FRED  Operator's  Manual 
[Perceptronics,  1988a,  1988b]  and  the  SIMNET  User's  Guide 
[U.S.  Army  Armor  Center,  1989]),  so  only  a  brief  description 
of  the  system  is  presented  here. 

In  general  terms,  AIRNET  is  a  team  training  simulator  in 
which  multiple  cockpits  are  networked  in  a  manner  that 
enables  the  crew  in  each  cockpit  to  interact  with  the  crews 
of  all  other  operational  cockpits.  Each  cockpit  has  a 
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computer  generated  display  that  portrays  topography,  other 
AIRNET  simulated  helicopters  (AH-64  or  OH-58D) ,  threat 
helicopters  (Hind  or  Havoc) ,  and  a  variety  of  ground  vehicles 
(both  friendly  and  threat) .  The  computer  generated  visual 
system  enables  crews  of  all  simulated  aircraft  to  view 
different  topography  and  targets  or  to  view  the  same 
topography  and  targets  simultaneously  from  different 
perspectives.  The  AIRNET  system  enables  a  crew  to  attack  a 
threat  individually  or  in  combination  with  the  crew  of  other 
simulated  aircraft;  conversely,  threats  can  return  fire  on 
AIRNET  crews'  aircraft  with  a  predetermined  level  of 
lethality . 

The  equations  of  motion  employed  simulate  a  generic 
helicopter,  so  the  handling  qualities  are  not  intended  to  be 
the  same  as  any  specific  helicopter. 

Figure  1  is  a  schematic  depiction  of  the  functional 
components  of  the  AIRNET  system.  Each  of  the  components  is 
described  briefly  below.  No  attempt  has  been  made  to  depict 
or  describe  the  individual  computers  employed  to  generate  the 
extra-cockpit  scene,  to  drive  the  simulated  aircraft,  or  to 
transfer  information  from  one  component  to  another. 

•  Simulator  Cockpits.  AIRNET  is  equipped  with  eight 
cockpits.  Each  cockpit  can  function  as  a  stand  alone 
unit,  but  ordinarily  is  linked  to  other  AIRNET 
cockpits  and  to  other  AIRNET  components.  The  design 
characteristics  of  the  cockpits  are  described  in  more 
detail  in  a  following  paragraph. 

•  Management/  Command,  and  Control... ..(MCC)  -Station .  The 
MCC  Station  provides  the  capability  for  activating, 
initializing,  and  deactivating  all  AIRNET  system 
components.  In  addition,  MCC  Station  computers 
simulate  (a)  indirect  fire  and  close  air  support,  and 
(b)  fuel  and  ammunition  automatic  resupply. 

•  Battle  Master  Station.  The  Battle  Master  Station 
provides  the  capability  to  specify  the  type,  number, 
and  initial  location  of  all  the  simulated  aircraft  and 
ground  vehicles  that  are  to  participate  in  a  training 
exercise.  The  Battle  Master  Station  is  also  used  to 
specify  the  initial  fuel  load  and  ammunition  load  for 
each  simulated  vehicle. 

•  Plan  View  Display  (PVD) .  The  PVD  provides  a  dynamic 
plan  view  of  the  simulated  operations  area  for  a 
training  exercise.  Associated  controls  provide  the 
capability  to  tailor  the  display  scale,  icon  size,  and 
display  content  to  meet  the  momentary  needs  of  the 
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Simulator  Cockpits 


Figure  1.  Main  components  of  AIRNET. 


user.  Ordinarily,  the  display  shows  a  contour  line 
depiction  of  terrain  relief  and  the  location  and  status 
of  air  and  ground  vehicles.  The  intervisibility  among 
vehicles  (and  other  intervehicle  information)  is 
displayed  on  demand. 

•  Stealth  Vehicle  Station.  The  Stealth  Vehicle  Station 
provides  the  capability  for  a  training  supervisor  or 
researcher  to  maneuver  to  and  observe  any  location 
within  the  simulated  operations  area  without  being 
observed  by  other  participants  or  by  simulator 
sensors . 

•  Tactical  Operations  Center  (TOC)  Station.  Close  air 
support  and  fire  support  are  controlled  from  the  TOC 
Station.  The  TOC  Station  provides  the  capability  to 
simulate  the  delivery  of  artillery  fire,  mortar  fire, 
and  close  air  support  bombs  to  designated  coordinates 
at  designated  times. 

•  Data  Logger  System  and  Data  Analysis  Station .  The 
Data  Logger  System  records  continuous  data  on  the 
location,  actions,  and  status  of  all  simulated 
vehicles.  The  data  can  be  recorded  on  an  internal 
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hard  disk  or,,  if  the  data  are  to  be  analyzed  or 
stored,  on  either  a  460  megabyte  floppy  disk  or  a  high 
density  magnetic  tape.  The  Data  Analysis  Station 
provides  the  capability  to  extract  selected  data  from 
the  floppy  disk  or  tape,  and  to  analyze  the  data  using 
standard  statistical  routines. 

•  Semi-Automated  Forces  (SAFOR)  Station.  As  the  name 
implies,  the  SAFOR  Station  is  used  to  specify  and 
control  automatically  the  air  and  ground  vehicles 
needed  for  a  force-on-force  training  exercise.  The 
computer  controlled  vehicles  can  be  aligned  as  either 
threat  or  friendly  vehicles.  In  short,  the  SAFOR 
Station  provides  the  capability  for  including  multiple 
simulated  vehicles  in  a  training  exercise  without  the 
need  for  an  individual  to  control  each  vehicle. 

All  AIRNET  cockpits  are  generic  cockpits  that  can  be 
configured  for  either  an  attack  helicopter  (tandem  seating) 
or  a  scout  helicopter  (side-by-side  seating) .  The  drawing  in 
Figure  2  shows  the  general  layout  of  an  AIRNET  cockpit.  The 
right-rear  station  is  the  pilot  station,  the  CPG  station  is 
in  front  of  the  pilot  station,  and  the  copilot/observer 
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station  is  to  the  immediate  left  of  the  pilot  station.  The 
computer-generated,  extra-cockpit  scene  is  shown  on  two  banks 
of  eight  displays  each.  One  bank  of  displays  is  located  in 
front  of  and  centered  on  the  pilot  station;  the  second  is 
located  in  front  of  and  centered  on  the  CPG  station.  The 
viewing  distance  to  the  displays  is  about  24  inches.  Since 
the  upper  row  has  two  fewer  displays  than  the  lower  row  (see 
Figure  2) ,  the  viewing  angle  is  less  on  the  upper  row  than  on 
the  lower  row  of  displays.  The  maximum  horizontal  and 
vertical  viewing  angle  is  145  degrees  and  80  degrees, 
respectively . 

Each  cockpit  station  is  equipped  with  only  the  controls 
and  instruments  that  are  considered  mission  essential  for  the 
crewmember  that  occupies  the  station;  hence,  each  crew  station 
is  configured  differently.  Table  2  lists  the  flight  controls, 
instrument  displays/ indicators,  and  switches  present  in  each  of 
the  three  cockpit  stations.  A  detailed  description  of  the 
design  and  function  of  the  crew  station  displays  and  controls 
is  found  in  the  AIRNET  operator's  manuals  for  the  scout  and  the 
attack  configurations  (Perceptronics,  1988a,  1988b) . 

The  evaluation  required  four  generic  AIRNET  devices, 
configured  as  either  an  OH-58C  or  an  AH-64  aircraft.  The 
devices  were  alternately  configured  to  represent  a  light 
attack  (one  scout  and  two  attack)  or  a  heavy  attack  (two  scout 
and  two  attack)  helicopter  team.  Semi-automated  threat  forces 
(Version  3.0)  were  employed  during  the  collective  task 
evaluation.  The  semi-automated  forces  provided  interactive 
air,  ground,  and  air  defense  artillery  (ADA)  threat  vehicles 
(Ceranowicz,  Downes-Martin,  &  Saffi,  1989;  Grignetti  &  Saffi, 
1989) .  This  capability  was  provided  from  Fort  Knox,  Kentucky, 
through  a  long-haul  network  via  a  56  kb  telephone  line. 


Tasks  Investigated 

As  was  stated  earlier,  the  research  investigated 
crewmembers'  ability  to  perform  both  individual  tasks2  and 
collective  tasks  in  AIRNET.  The  13  individual  flying  tasks 
selected  for  study  are  listed  on  page  10.  The  individual 
tasks  selected  must  be  performed  frequently  and  effectively 
to  perform  collective  tasks.  All  individual  task  definitions 
and  performance  standards  are  as  specified  in  the  appropriate 
ATM . 


2The  tasks  referred  to  in  this  report  as  individual  tasks  are 
often  referred  to  elsewhere  as  ATM  tasks.  The  term  is  used 
here  to  ensure  a  clear  distinction  between  individual  and 
collective  tasks. 
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•  Hovering  Flight 

•  Nap-of-the-Earth  (NOE)  Flight 

•  Contour  Flight 

•  Low  Level  Flight 

•  Acceleration 

•  Normal  Deceleration 

•  NOE  Deceleration 

•  High  Speed  Flight 

•  Vertical  Masking/Unmasking 

•  Lateral  Masking/Unmasking 

•  Navigation:  airspeed  20  -  50  knots 

•  Navigation:  airspeed  50  -  80  knots 

•  Navigation:  airspeed  80  -  110  knots 

In  general  terms,  the  collective  tasks  investigated 
consist  of  the  tasks  that  members  of  a  helicopter  attack  team 
must  perform  to  accomplish  representative  missions.  The 
tasks  were  performed  in  the  context  of  a  tactically  and 
doctrinally  valid  scenario  for  each  of  three  types  of 
missions:  a  cross-FLOT  (forward  line  of  troops)  deliberate 

air  attack  mission,  a  deliberate  offensive  attack  mission, 
and  a  hasty  attack  mission.  The  collective  tasks  performed 
during  one  or  more  of  the  mission  scenarios  are  listed  below. 
Appendix  A  contains  a  detailed  listing  of  the  collective 
tasks  and  subtasks  investigated  for  both  the  scout  and  the 
attack  crewmembers . 

•  Conduct  Movement  to  Holding  Area 

•  Use  Passive  Air  Defense  Measures 

•  Conduct  Tactical  Air  Movement  as  Part  of  a  Movement  to 
Contact 

•  Move  to  and  Occupy  a  Battle  Position 

•  Use  Countermeasures  Against  Enemy  Air  Defense  Artillery 

•  Report  Intelligence  Data 

•  Establish  Contact 

•  Engage  Targets 

•  Conduct  Deliberate  Air  Attack 

•  Conduct  Hasty  Air  Combat  Operations 

•  Return  to  Assembly  Area  and  Prepare  for  Future 
Operations 

•  Move  From  a  Battle  Position 

•  Conduct  Joint  Air  Attack  Team  (JAAT)  Operations 

•  Conduct  Downed  Aircrew  Recovery 
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Performance  Assessment  Data 


Two  types  of  performance  assessment  data  were  collected 
during  this  research.  First,  task  performance  ratings  were 
obtained  from  SMEs  judging  the  adequacy  with  which  each 
individual  and  collective  task/subtask  was  performed. 

Similar  ratings  of  collective  task  performance  were  obtained 
from  the  participating  crewmembers .  Second,  crewmembers  were 
required  to  complete  questionnaire  items  designed  to  assess 
crewmembers'  opinions  about  the  adequacy  of  AIRNET  components 
for  performing  and  training  selected  tasks.  The  task 
performance  ratings  and  the  questionnaires  are  discussed 
below . 


Task  Performance  Ratings. 

Participants  were  required  to  rate  the  adequacy  with 
which  each  individual  task  and  each  collective  task  was 
performed.  SMEs  rated  the  performance  of  individual  tasks; 
both  SMEs  and  crewmembers  rated  the  performance  of  collective 
tasks.  A  two  step  rating  procedure  was  used.  First, 
participants  were  required  to  rate  task  performance  as 
adequate  or  inadequate.  Performance  was  rated  as  adequate  if 
the  task  being  evaluated  was  performed  to  the  standards 
specified  in  the  appropriate  ATM  (individual  tasks)  or 
standards  that  are  used  to  assess  performance  adequacy  in  the 
aircraft  (collective  tasks) .  Second,  if  task  performance  was 
judged  inadequate,  the  participant  was  required  to  judge 
whether  the  inadequate  performance  was  due  principally  to  a 
crew  skill  deficiency  or  to  an  AIRNET  system  shortcoming. 

For  each  collective  task,  performance  was  rated  for 
(a)  the  composite  task  and  (b)  the  subtasks  that  comprise  the 
task.  Appendix  B  presents  specimens  of  the  rating  forms  used 
to  rate  collective  tasks.  The  specimen  rating  forms 
illustrate  the  breakout  of  subtasks  for  three  different 
collective  tasks. 


Target  Detection/Identification  Range  Measures 

Measures  were  obtained  of  the  maximum  range  at  which 
crewmembers  could  detect  and  identify  targets  with  unaided 
vision  and  with  the  AIRNET  FLIR. 


Other  Crewmember  Ratings 

Questionnaires  were  used  to  assess  crewmembers' 
judgments  about  (a)  the  adequacy  of  specific  AIRNET 
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components  for  performing  specific  tasks,  (b)  the  maximum 
range  at  which  targets  can  be  detected  (direct  view  and 
FLIR) ,  (c)  the  physiological  discomfort  and  simulator 

sickness  symptoms  experienced  in  AIRNET,  and  (d)  the  probable 
training  effectiveness  of  AIRNET.  Two  different 
questionnaires  (see  Appendixes  C  and  D)  were  completed  by  all 
crewmembers .  One  questionnaire  (Appendix  C)  was  completed 
twice — once  early  and  once  late  in  the  research;  the  second 
questionnaire  (Appendix  D)  was  completed  at  an  intermediate 
point  in  time.  The  specific  times  at  which  the 
questionnaires  were  completed  are  described  in  the  Procedures 
section  of  this  report. 

Questionnaire  items  designed  to  assess  crewmembers' 
judgments  about  AIRNET  component  adequacy  required 
crewmembers  to  use  a  5-point  rating  scale  to  rate  the 
adequacy  of  a  specific  AIRNET  component /attribute  (e.g. 
visual  system  f ield-of-view)  for  performing  each  of  a  set  of 
tasks  (e.g.,  hovering,  forward  flight,  target  acquisition, 
etc.) .  The  AIRNET  components/attributes  for  which  adequacy 
ratings  were  obtained  include  (a)  the  computer-generated, 
extra-cockpit  display;  (b)  the  communication  system;  (c)  the 
handling  qualities  (flying  characteristics)  of  the  simulated 
aircraft;  (d)  the  responsiveness  of  the  simulation  to  control 
inputs;  (e)  the  cockpit  controls,  switches,  displays,  and 
indicators;  and  (f)  the  sound  generation  system.  The  verbal 
anchors  for  the  5-point  rating  scale  are  as  follows: 

•  1  =  totally  inadequate, 

•  2  =  somewhat  inadequate, 

•  3  =  just  adequate 

•  4  =  more  than  adequate,  and 

•  5  =  much  more  than  adequate. 

A  5-point  rating  scale  with  different  verbal  anchors 
(see  Appendix  D)  was  used  to  assess  crewmembers'  judgments 
about  AIRNET' s  potential  value  for  training  each  of  a  set  of 
specific  tasks.  The  other  questionnaire  items  were  yes/no 
items  or  checklist  items  (e.g.  maximum  target  detection 
range)  . 


Automated  Performance  Measures 

The  Data  Logger  System  was  used  to  record  data 
continuously  throughout  each  session  and  to  compute  selected 
summary  statistics  at  the  end  of  each  session.  These  data 
were  used  by  the  research  psychologist  to  augment  the  SME  and 
crewmember  raters'  direct  observations  of  performance  of 
collective  tasks.  However,  the  data  were  not  suitable  for 
deriving  objective  measures  of  performance  on  specific  tasks. 
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Procedures 


The  flow  diagram  in  Figure  3  is  a  schematic  depiction  of 
the  procedures  employed  to  accomplish  the  research.  The 
blocks  in  Figure  3  depict  the  sequence  of  research  tasks 
accomplished;  the  ovals  depict  the  data  generated  by  the 
associated  research  task.  The  numbers  to  the  left  of  each 
block  specify  days  on  which  the  associated  research  task  was 
accomplished.  Each  of  the  research  tasks  is  described  briefly 
below . 


Administer  Crewmember  Instruction 

All  crewmembers  received  3  hours  of  classroom 
instruction  on  the  design  characteristics  and  operation  of 
AIRNET.  The  classroom  instruction  was  followed  by  6  hours  of 
training  in  the  AIRNET  cockpit.  The  in-cockpit  instruction 
was  designed  to  familia-ize  crewmembers  with  the  location  and 
function  of  all  displays  and  controls  present  in  the  station 
the  crewmember  was  to  occupy  during  the  research. 

Crewmembers  selected  to  occupy  the  pilot  station  were 
instructed  on  the  handling  characteristics  of  the  simulated 
aircraft  and  were  given  an  opportunity  to  practice  a  wide 
range  of  flying  tasks.  All  crewmember  instruction  was 
accomplished  during  Days  1  through  8. 


Administer  Crewmember  Questionnaire 

Upon  completion  of  the  9  hours  of  instruction,  each 
crewmember  was  required  to  complete  Crewmember  Questionnaire 
#1  (Ql)  for  the  first  time.  (See  earlier  discussion  and 
Appendix  C  for  details  about  the  questionnaire.)  All 
crewmembers  completed  the  first  administration  of 
Questionnaire  #1  on  Days  9  and  10. 


Assess  Individual  Task  Performance  and  Target  Detection/ 
Identification  Range 

Crewmembers  who  occupied  the  pilot  station  were  assessed 
on  their  ability  to  perform  each  of  the  13  individual  tasks 
listed  earlier.  Each  aviator  performed  each  individual  task 
three  consecutive  times.  An  aviator's  performance  was  rated 
as  adequate  if  he  performed  the  task  to  the  standards 
specified  in  the  appropriate  ATM  on  two  consecutive  trials. 
Otherwise,  the  aviator's  performance  was  rated  as  inadequate. 
All  evaluations  were  performed  by  a  highly  experienced  IP. 
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Days  1  -  8 


Days  9-10 


Days  9-10 


Days  11-20 


Days  21-22 


Days  21-22 


Days  21-22 


Figure  3.  AIRNET  training  and  evaluation  procedure 


During  the  same  session  in  which  individual  task 
performance  was  assessed,  tests  were  conducted  to  determine 
the  maximum  range  at  which  targets  can  be  detected  and 
identified  with  unaided  vision  and  with  the  AIRNET  FLIR.  All 
crewmembers  participated  in  the  tests  of  target  detection/ 
identification  range  with  unaided  vision;  only  the  crews  of 
the  attack  aircraft  participated  in  the  tests  conducted  with 
the  AIRNET  FLIR.  For  all  tests,  the  simulated  aircraft 
remained  at  a  stationary  location  on  the  computer  generated 
terrain  at  a  fixed  altitude  of  50  ft.  Targets  were  located 
within  a  3,000  m  wide  corridor  directly  ahead  of  the 
stationary  aircraft;  the  longitudinal  axis  of  the  simulated 
aircraft  was  aligned  with  the  center  of  the  corridor. 

The  targets  were  positioned  at  intervals  of  1,000  m  from 
the  simulated  aircraft's  location.  That  is,  the  nearest 
target  was  located  1,000  m  from  the  aircraft's  position,  the 
next  target  was  located  2,000  m  from  the  aircraft's  position, 
and  so  on.  The  lateral  position  of  the  targets  within  the 
corridor  was  varied  randomly,  but  always  remained  within  the 
3,000  m  wide  corridor.  The  most  distant  target  used  in  the 
tests  with  unaided  vision  was  located  at  a  range  of  4,000  m; 
the  most  distant  target  used  in  the  tests  with  the  AIRNET 
FLIR  was  located  at  a  range  of  8,000  m. 

The  targets  used  in  the  tests  included:  tanks,  armored 
personnel  carriers,  ammunition  trucks,  fuel  trucks,  and 
howitzers.  The  targets  were  alternately  configured  as 
friendly  or  enemy.  Color  coding  alone  was  used  to  designate 
a  target  as  friendly  or  enemy.  Any  brown  target  was 
designated  a  friendly  and  any  green  target  was  designated  an 
enemy.  Hence,  identification  of  a  target  as  friendly  or 
enemy  required  only  a  color  discrimination  (brown  or  green) . 
All  crewmembers  had  viewed  all  targets  from  various  ranges, 
including  close  ranges,  before  commencing  the  test. 

Crewmembers  were  instructed  to  employ  normal  scan 
procedures  in  searching  for  targets  along  a  corridor  1,500  m 
to  either  side  of  the  longitudinal  axis  of  the  aircraft. 

They  were  told  that  more  than  one  target  was  located  within 
the  corridor,  but  they  were  not  told  the  exact  number  of 
targets  present.  When  a  crewmember  reported  having  detected 
a  target,  he  was  asked  to  identify  the  target  as  friendly  or 
enemy.  If  the  crewmember  correctly  identified  the  target  as 
friendly  or  enemy,  he  was  asked  to  identify  the  specific 
target  type  (e.g..  Ml  tank).  Target  detection  ranges  and 
target  identification  responses  were  recorded  on  a  data 
sheet.  The  above  procedure  was  repeated  until  the  crewmember 
reported  that  he  had  detected  and  identified  all  targets 
present  in  the  corridor. 
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Assess  Collective  Task  Performance 


The  collective  task  evaluation  was  conducted  on  training 
Days  11  through  20.  The  company  commander  organized  the 
scout  and  attack  crews  into  two  teams,  light-  and  heavy- 
attack.  The  light  team.  Team  1,  consisted  of  one  scout  and 
two  attack  crews;  the  heavy  team.  Team  2,  consisted  of  two 
scout  and  two  attack  crews.  The  company  commander  briefed 
each  mission  and  participated  as  the  battle  captain  from  the 
tactical  operations  center.  Both  teams  received  the  same 
mission  briefing  and  were  exposed  to  the  same  threat  arrays 
for  each  session.  Table  3  shows  the  types  of  missions  flown 
during  the  collective  task  evaluation  and  the  type  and 
lethality  of  threats  encountered  during  each  evaluation 


Table  3 

Mission,  Type  Threat,  and  Lethality  Used  During  Five  Sessions 
of  Collective  Task  Evaluation 


Mission 

Deliberate 

attack 

Deliberate  air  Hasty  air 

attack:  cross-FLOT  attack 

Training 
days  11-12 

Ground  vehicles; 
passive 

Training 
days  13  -  14 

Ground  and  air 
vehicles;  passive 

Training 
day3  15  -  16 

Ground  vehicles; 

Lethality: 
novice/competent 
Range:  5000  m 

Training 
days  17  -  18 

Air  vehicles 

Air  ratio:  (2:1) 

Lethality:  novice /competent 

Range:  5000  m 

Training 
day3  19  -  20 

Ground  and  air  vehicles 
Air  ratio:  (1:1) 
Lethality:  competent 

Range:  5000  m 

Note .  FLOT  =  forward  line  of  troops. 
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session.  The  teams  encountered  only  passive  ground  and  air 
vehicles  during  the  first  4  days  of  the  evaluation.  The 
remaining  sessions  were  conducted  against  semi-automated 
threat  vehicles. 

Each  team  encountered  only  ground  vehicles  during  the 
deliberate  attack  mission  against  the  semi-automated  threat. 
The  enemy  ADA  was  initiated  at  the  novice  level  of  lethality 
(probability  of  a  hit  =  .10)  at  a  range  of  5,000  m.  If  the 
first  engagement  of  the  first  session  was  successful,  the 
threat  lethality  was  increased  to  the  competent  level 
(probability  of  a  hit  =  .33) . 

Enemy  air  vehicles  were  encountered  during  the  cross- 
FLOT  deliberate  air  attack  mission.  The  lethality  was 
initiated  at  the  novice  level;  if  the  initial  engagement  was 
successful,  the  lethality  of  the  vehicles  was  increased  to 
the  competent  level  for  the  following  engagement.  The 
density  of  the  threat  array  for  the  deliberate  air  attack 
mission  was  two  to  one;  that  is,  twice  as  many  air  threat 
vehicles  were  present.  ;:han  simulated  aircraft. 

The  teams  encountered  both  enemy  air  and  ground  vehicles 
during  the  hasty  air  attack  mission.  The  lethality  of  both 
air  and  ground  vehicles  was  set  at  the  competent  level  during 
the  hasty  attack  mission. 

Team  aircraft  that  crashed  during  the  conduct  of  a 
mission  were  reconstituted  and  allowed  to  continue  the 
mission.  The  aircraft  were  reinitialized  at  the  holding  area 
with  full  fuel  and  ammunition  loads. 

SME  in-simulator  performance  ratings  and  postmission 
crew  ratings  were  collected  for  each  session.  SMEs  judged 
the  performance  of  crewmembers  on  mission  subtasks  as  they 
occurred  during  the  mission.  The  composite  mission 
performance  rating  was  completed  by  the  SMEs  at  the  end  of 
each  session.  Judgment  about  adequate  composite  task 
performance  was  based  on  the  number  and  critical  nature  of 
the  subtasks  that  were  adequately  performed  for  each  task. 

The  critical  subtasks  were  identified  by  all  SMEs  as  being 
both  necessary  and  sufficient  to  accomplish  the  composite 
task.  A  debriefing  was  conducted  by  the  company  commander 
and  the  SME  raters  following  each  session. 


Administer  Crewmember  Questionnaire 

Following  the  final  collective  task  evaluation  session, 
all  participants  were  required  to  complete  Crewmember 
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Questionnaire  #2  (Q2,  see  Appendix  D) .  All  crewmembers 
completed  Q2  on  Days  21  and  22. 


Assess  Individual  Task  Performance  and  Target  Detection/ 
Identification  Range 

On  Days  21  and  22,  the  crewmembers  were  assessed  a 
second  time  on  their  ability  to  perform  individual  tasks  and 
to  detect  and  identify  targets.  The  same  procedures 
described  for  the  first  assessment  were  employed  in  the 
second  iteration.  The  second  assessment  was  conducted  to 
determine  the  effect  of  the  experience  gained  during  the 
collective  task  evaluation  sessions  on  target  detection  and 
identification  performance. 


Administer  Crewmember  Questionnaire 

The  final  task  was  to  administer  Q1  a  second  time.  The 
pilots  and  the  aeroscout  observers  were  required  to  complete 
Q1  a  second  time  to  determine  the  effect  of  the  experience 
gained  during  the  collective  task  evaluation  sessions  on 
their  subjective  ratings.  Q1  was  administered  on  Days  21  and 
22. 


Assess  AIRNET. Design-Short comings 

About  2  months  after  completing  the  data  collection 
procedure  described  above,  two  of  the  SMEs  who  evaluated  the 
crewmembers'  performance  on  collective  tasks  participated  in 
a  post  hoc  assessment  intended  to  identify  the  AIRNET 
shortcomings  that  contribute  to  collective  task  performance 
deficiencies.  Each  SME  was  provided  with  two  types  of  data 
(compiled  during  the  collective  task  evaluation) .  First, 

SMEs  were  provided  a  list  of  the  AIRNET  shortcomings 
identified  by  SMEs  as  probable  contributors  to  collective 
task  performance  difficulties.  Second,  SMEs  were  provided  a 
table  showing  the  collective  tasks  for  which  performance  was 
judged  inadequate  by  SMEs  or  crewmembers  on  two  or  more 
iterations.  The  SMEs  were  instructed  to  (a)  consider  each 
collective  task  for  which  performance  was  judged  inadequate, 
(b)  identify  the  AIRNET  design  shortcomings  listed  that 
contribute  to  the  difficulty  in  performing  the  task,  and  (c) 
identify  design  shortcomings,  other  than  those  listed,  that 
contribute  to  the  difficulty  in  performing  the  task.  The  two 
SMEs  were  instructed  to  collaborate  as  necessary  to  identify 
AIRNET  shortcomings  on  which  both  agreed.  The  judgments  were 
recorded  by  a  member  of  the  research  staff. 
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Results 


This  section  of  the  report  describes  the  research 
results.  It  commences  with  a  description  of  crewmembers' 
performance  on  the  individual  flying  tasks  and  on  the  target 
detection/identification  task.  Next,  crewmembers' 
performance  on  the  collective  tasks  is  described.  The 
remainder  of  the  section  describes  crewmembers'  responses  to 
the  questionnaire  items . 

Most  of  the  results  are  presented  in  the  form  of  two¬ 
fold  graphs  (or  tables) .  Typically,  the  graphs  depict  the 
proportion3  of  crewmembers  who  responded  in  a  certain  manner 
as  a  function  of  a  task  or  equipment  variable.  For  instance, 
there  are  graphs  that  show  the  proportion  of  crewmembers  who 
performed  adequately  on  each  of  a  variety  of  tasks.  There 
are  other  graphs  that  show  the  proportion  of  crewmembers  who 
wei,  able  to  detect  or  identify  a  target  from  each  of  a 
variety  of  viewing  ranges.  There  are  still  other  graphs  that 
show  the  proportion  of  crewmembers  who  selected  each  value  of 
a  rating  scale  when  rating  the  adequacy  of  AIRNET  for 
performing  each  of  a  variety  of  tasks.  The  interpretation  of 
these  data  requires  an  assessment  of  either  (a)  the  absolute 
magnitude  of  one  or  more  proportions  or  (b)  the  magnitude  of 
the  difference  between  two  proportions. 

In  the  present  research,  the  most  common  purpose  for 
assessing  the  absolute  magnitude  of  a  single  proportion  (or 
set  of  proportions)  is  to  determine  whether  AIRNET  is 
adequate  for  a  given  training  application.  This  type 
assessment  is  necessarily  subjective.  For  instance,  consider 
the  finding  that  approximately  85%  of  the  aviators  were  able 
to  perform  the  vertical  masking  task  to  standards.  Does  this 
finding  suggest  that  AIRNET  is  adequate?  What  if  the  value 
had  been  50%  rather  than  85% ?  The  authors  know  of  no  purely 
objective  methods  for  answering  such  questions.  So,  although 
the  authors  have  made  such  assessments  in  this  report,  the 
reader  must  keep  in  mind  that  the  assessments  are  subjective 
and  that,  in  some  instances,  others  may  interpret  the 
findings  somewhat  differently. 

This  research  was  designed  to  yield  descriptive  data 
rather  than  data  with  which  to  assess  the  statistical 
significance  of  difference  between  groups,  conditions,  and  so 
on.  However,  there  are  tests  to  determine  whether  the 
difference  between  two  proportions  represents  a  true 

throughout  this  report,  the  term  proportion  is  used  in 
discussing  statistical  tests  and  the  term  percentage  is  used 
in  describing  specific  findings. 


19 


(statistically  reliable)  difference  or,  conversely,  a 
difference  that  is  attributable  to  sampling  variability.  The 
authors  have  tested  and  reported  the  statistical  significance 
of  differences  between  selected  proportions  when  it  was 
judged  meaningful  to  do  so.  The  method  used  to  test  for  the 
statistical  significance  between  proportions  is  a  normal 
curve  test  applied  to  the  arcsine  transformation  of  the 
proportions;  the  test  is  described  in  detail  by  Cohen  (1977, 
pp .  179-211).  The  test  was  selected  because  it  is  less 
biased  by  small  sample  sizes  than  alternative  tests.  Cohen's 
method  is  recommended  for  use  by  readers  who  wish  to  test 
differences  not  tested  by  the  authors. 

Table  4  was  prepared  to  facilitate  readers'  assessment 
of  the  statistical  significance  of  differences  between 
proportions .  Table  4  shows  the  minimum  difference  between 
proportions  that  is  required  to  reach  the  .05  level  of 
significance  (two  tailed  test)  as  a  function  of  the  sample 
size  and  the  smallest  of  the  two  proportions  being  compared. 
Examination  of  the  values  in  Table  4  shows  that  there  is  an 
inverse  relationship  between  the  size  of  the  sample  and  the 
size  of  the  difference  required  to  reach  statistical 
significance.  For  very  small  Ns,  the  size  of  the  difference 
required  to  reach  statistical  significance  is  very  large  (as 
much  as  .64  when  n  =  4  and  Pi  =  .2) . 

The  reader  should  keep  in  mind  that  a  small  sample  size 
increases  the  probability  of  erroneously  concluding  that 
there  are  no  statistically  significant  differences  between 
proportions  (Type  II  error) .  This  means  that  some  « 

differences  that  fail  to  reach  the  critical  values  shown  in 
Table  4  are,  in  fact,  true  differences.  Or,  stated 
differently,  a  difference  that  fails  to  reach  the  critical 
value  would  prove  to  be  statistically  significant  if  the 
research  was  repeated  with  a  larger  sample.  Unfortunately, 
high  probability  of  Type  II  errors  is  the  price  that  must  be 
paid  for  the  economy  of  a  small  sample  size,  regardless  of 
the  statistic  that  is  used  to  assess  the  difference. 


Performance  on  Individual  Flying  Tasks 

The  results  of  the  assessment  of  aviators'  performance 
on  individual  tasks  are  summarized  in  Figure  4 .  The  bars 
show,  by  task,  the  percentage  of  iterations  performed  to 
standards.  The  cross-hatched  bars  depict  performance  on  the 
first  assessment  (days  9  -  10) ;  the  solid  bars  depict 
performance  on  the  second  assessment  (days  21  -  22) .  Each  of 
seven  aviators  performed  each  task  three  times,  so  the 
percentage  values  are  based  on  21  iterations. 
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TASK  NAME 

Hovering  Flight 
Acceleration 
Navigation  80-110  Knots 
Navigation  50-80  Knots 
Navigation  20-50  Knots 
Nap-of-the-Earth  Flight 
Low  Level  Flight 
High  Speed  Flight 
Contour  Flight 
Vertical  Masking 
Lateral  Masking 
Normal  Deceleration 
Nap-of-the-earth  Deceleration 


0  20  40  60  80  100 


Percentage  (21  Iterations) 


Figure  4.  Percentage  of  task  iterations  performed  to 
standards  during  the  initial  and  final  individual  task 
evaluations . 


Figure  4  shows  that  aviators  performed  9  of  the  13  tasks 
to  standards  on  at  least  85%  of  the  iterations  (see  first  9 
tasks  listed) .  For  5  of  the  9  tasks,  fewer  iterations  were 
performed  to  standard  on  the  final  (85%)  than  the  initial 
(100%)  assessment.  However,  the  difference  is  not 
statistically  significant  [P ^  -  p2  = . 143) > . 05 ]  .  Furthermore, 
there  are  no  reasons  to  expect  performance  on  the  final 
assessment  to  be  poorer  than  performance  on  the  initial 
assessment.  Hence,  the  following  conclusions  seem  warranted 
for  the  first  9  tasks  listed  in  Figure  4 : 

•  the  difference  between  performance  on  the  initial  and 
final  assessments  represents  sampling  error  or  rater 
bias,  and 

•  the  tasks  can  be  performed  to  standards  at  least  85% 
of  the  time  they  are  attempted. 

There  are  four  tasks  for  which  performance  appears  to  be 
a  potentially  serious  problem.  On  the  initial  assessment, 
vertical  and  lateral  masking  were  performed  to  standards  on 
only  57%  of  the  iterations;  normal  deceleration  and  NOE 
deceleration  were  performed  to  standards  on  only  43%  of  the 
iterations.  Although  performance  on  masking  (vertical  and 
lateral)  and  normal  deceleration  improved  with  practice,  only 
"71%  of  the  iterations  of  these  tasks  were  performed  to 
standards  on  the  final  assessment.  Performance  of  NOE 
deceleration  was  no  better  on  the  final  than  the  initial 
assessment  (43%  of  iterations  performed  to  standards). 
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All  the  aviators  who  participated  in  the  research 
routinely  perform  lateral  masking,  normal  deceleration,  and 
NOE  deceleration  in  the  aircraft.  So,  the  difficulty  in 
performing  these  tasks  in  AIRNET  can  probably  be  attributed 
to  design  shortcomings  in  one  or  more  of  AIRNET' s  components. 
Interviews  with  crewmembers  and  crewmembers'  responses  to 
questionnaire  items  indicate  that  the  difficulty  stems  mainly 
from  the  handling  qualities  of  the  AIRNET  simulated  aircraft. 
The  crewmembers  reported  that  the  simulated  aircraft  simply 
does  not  fly  like  a  helicopter  and  that  the  control  inputs 
required  to  decelerate  in  a  helicopter  are  not  effective  in 
decelerating  the  AIRNET  simulated  aircraft. 


Performance  on  Target  Detection  and  Identification 

Figure  5  shows  the  percentage  of  targets  detected  and 
identified  with  unaided  vision  (AIRNET  extra-cockpit  display 
alone)  as  a  function  of  viewing  range.  Performance  on  the 
initial  assessment  is  shown  in  the  upper  graph;  performance 
on  the  final  assessment  is  shown  in  the  lower  graph. 
Individual  bars  depict  the  percentage  of  successful  target 
detections  (solid  bars) ,  identifications  as  friend  or  foe 
(cross-hatched  bars),  and  specific  identifications  (stippled 
bars) .  All  percentage  values  are  based  on  a  total  of  seven 
opportunities  for  a  correct  detection  and  identification  at 
each  viewing  range  (one  trial  by  each  of  seven  crewmembers 
and  one  target  at  each  viewing  range) . 

All  targets  at  a  viewing  range  of  1,000  m  were  detected 
and  correctly  identified  (see  Figure  5) .  However,  the 
percentage  of  detections  and  correct  identifications 
decreased  dramatically  at  a  viewing  range  of  2,000  m.  In  the 
initial  assessment,  only  57%  of  the  targets  at  a  range  of 
2,000  m  were  detected;  only  29%  were  correctly  identified  as 
friend  for  foe,  and  only  14%  were  identified  by  specific 
type.  No  targets  were  detected  or  identified  from  a  viewing 
range  of  3,000  m  or  more.  There  was  no  statistically 
significant  performance  improvement  on  the  final  assessment 
at  any  viewing  range . 

Figure  6  shows  the  percentage  of  targets  detected  and 
identified  with  the  AIRNET  FLIR  as  a  function  of  viewing 
range.  The  format  and  coding  of  data  shown  in  Figure  6  are 
the  same  as  that  described  above  for  Figure  5.  However,  the 
percentage  values  in  Figure  6  are  based  on  a  total  of  only 
four  opportunities  for  correct  detection  and  identification 
at  each  viewing  range.  The  data  indicate  that  targets  can  be 
detected  reliably  (close  to  100%  accuracy)  with  the  AIRNET 
FLIR  to  a  range  of  6,000  m.  Identification  of  a  target  as  a 
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INITIAL  ASSESSMENT 


Viewing  Range  (Meters) 


FINAL  ASSESSMENT 


Viewing  Range  (Meters) 


Figure  5.  Percentage  of  targets  detected,  identified  as 
friend  or  foe  (IFF),  and  identified  by  type  (ID)  with 
unaided  vision  as  a  function  of  viewing  range  during  the 
initial  and  final  assessments. 


friend  or  foe  can  be  accomplished  reliably  (no  more  than  25% 
errors)  with  the  AIRNET  FLIR  to  a  range  of  4,000  m.  Beyond 
4,000  m,  IFF  performance  drops  to  near  chance  level  (50%  of 
targets  detected) . 

Specific  target  identification  can  be  accomplished 
reliably  using  AIRNET  FLIR  at  a  range  of  1,000  m,  but  the 
frequency  of  correct  (specific)  identifications  is 
considerably  less  at  ranges  beyond  1,000  m.  On  the  initial 
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INITIAL  ASSESSMENT 


Viewing  Range  (Meters) 


|  Detection 
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Figure  6.  Percentage  of  targets  detected  and  identified 
with  the  AIRNET  FLIR  as  a  function  of  viewing  range  during 
the  initial  and  final  assessments. 


assessment,  75%  of  the  targets  were  identified  correctly 
(specific  identification)  at  ranges  of  2,000  and  3,000  m;  50% 
were  identified  correctly  at  a  range  of  4,000  m;  only  one 
target  was  identified  correctly  at  a  range  of  5,000  m  or 
more.  Performance  on  specific  target  identification  did  not 
improve  on  the  final  assessment;  in  fact,  performance  was  the 
same  or  poorer  at  all  ranges  beyond  2,000  m.  Although  the 
size  of  the  sample  is  small,  the  data  support  the  following 
conclusions  about  specific  target  identification  with  AIRNET 
FLIR: 
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•  all  or  nearly  all  targets  can  be  identified  correctly 
at  a  range  of  1,000  m  or  less, 

•  between  50%  and  75%  of  targets  can  be  identified 
correctly  at  ranges  between  2,000  and  4,000  m, 

•  25%  or  fewer  targets  can  be  identified  correctly  at  a 
range  of  5,000  m,  and 

•  few,  if  any,  targets  can  be  identified  correctly  at 
ranges  beyond  5,000  m. 

For  both  unaided  vision  and  FLIR,  the  maximum  range  at 
which  targets  can  be  detected  and  correctly  identified  is 
less  in  AIRNET  than  under  typical  operational  conditions. 
(See  the  Discussion  and  Conclusions  section  for  a  comparison 
of  target  detection  and  identification  ranges  in  AIRNET  and 
in  an  aircraft.)  Therefore,  the  effectiveness  of  collective 
training  in  AIRNET  will  be  degraded  to  the  extent  that 
training  is  dependent  upon  the  ability  to  operate  in  the 
simulated  environment  at  target  ranges  comparable  to  the 
ranges  at  which  helicopter  teams  are  expected  to  function  in 
a  combat  environment . 


Performance  on  Collective  Tasks 

This  subsection  describes  the  data  that  bear  on  the 
performance  of  collective  tasks  in  AIRNET.  Two  types  of  data 
are  presented  and  described.  First,  data  are  presented  that 
identify  the  collective  tasks  for  which  (a)  performance  was 
judged  inadequate  and  (b)  inadequate  performance  was 
attributed  to  an  AIRNET  design  shortcoming.  The  data  on 
collective  task  performance  adequacy  include  the  judgments  of 
both  the  SME  evaluators  and  the  participating  crewmembers. 
Second,  data  are  presented  that  identify  the  specific  AIRNET 
shortcomings  that  SMEs  judged  to  be  contributors  to  the 
inadequate  performance  of  one  or  more  collective  tasks. 

Most  of  the  collective  tasks  investigated  consist  of 
several  different  subtasks.  Participants  were  required  to 
judge  AIRNET' s  adequacy  for  performing  each  of  the  subtasks 
that  comprised  the  collective  tasks.  The  data  on  performance 
adequacy  are  presented  by  subtask. 


Collective  Task  Performance  Adequacy 

The  findings  show  that  there  was  a  general  lack  of 
uniformity  in  the  judgments  of  AIRNET' s  adequacy  for 
performing  collective  subtasks.  That  is,  there  were  few 
subtasks  for  which  performance  was  consistently  judged 
adequate  (or  inadequate)  from  one  iteration  to  another. 
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Furthermore,  the  judgments  of  SMEs  and  crewmembers  often 
differed  substantially. 

The  lack  of  uniformity  in  judgments  appears  to  stem  from 
two  sources.  First,  casual  observations  during  the 
evaluation  sessions  suggest  that  most  of  the  SME  evaluators 
and  crews  employed  performance  standards  that  are  less 
demanding  than  the  standards  employed  to  evaluate  collective 
performance  in  an  aircraft.  Second,  the  judgments  of 
performance  adequacy  by  crewmembers  appear  to  be  influenced 
significantly  by  the  amount  of  effort  expended  in  performing 
a  subtask.  That  is,  performance  on  subtasks  that  require 
extraordinary  effort  by  crewmembers  was  sometimes  judged 
inadequate  by  crewmembers  even  though  SMEs  judged  performance 
to  be  adequate.  It  is  the  authors'  opinion  that  these  biases 
resulted  in  an  underestimation  of  the  proportion  of 
iterations  for  which  performance  was  inadequate. 

To  focus  on  problem  subtasks,  the  data  were  examined  to 
identify  subtasks  for  which  performance  was  judged  inadequate 
on  at  least  two  iterations  and  the  inadequacy  was  attributed 
to  an  AIRNET  system  shortcoming.  The  iteration  criterion 
could  be  met  in  any  one  of  three  ways:  two  iterations  judged 
inadequate  by  SMEs,  two  iterations  judged  inadequate  by 
crews,  or  one  iteration  judged  inadequate  by  both  an  SME  and 
a  crew. 

The  35  subtasks  of  16  collective  tasks  that  met  these 
criteria  are  listed  in  the  left-hand  column  of  Table  5.  The 
second  and  third  columns  in  Table  5  show,  by  subtask,  the 
percentage  of  iterations  judged  inadequate  by  SMEs  and  crews, 
respectively.  For  the  18  subtasks  performed  by  both  the 
attack  and  the  scout  crews,  the  percentage  values  are  based 
on  seven  iterations.  For  subtasks  performed  by  only  attack 
crews  or  by  only  scout  crews,  the  numbers  of  iterations  are 
four  and  three,  respectively. 

A  detailed  discussion  of  these  findings  is  presented  in 
the  following  subsection,  in  conjunction  with  data  on  the 
AIRNET  shortcomings  judged  to  be  contributors  to  performance 
inadequacies.  At  this  point,  however,  it  should  be  noted 
that  (a)  neither  SMEs  nor  crews  judged  performance  on  any 
subtask  to  be  inadequate  on  more  than  67%  of  the  iterations, 
(b)  performance  on  14  subtasks  was  judged  inadequate  by 
crewmembers  but  not  SMEs,  and  (c)  SMEs  judged  performance 
inadequate  more  often  than  crewmembers  on  only  six  subtasks. 
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Table  5 


Collective  Subtasks  for  Which  Performance  Inadequacy  was 
Attributed  to  an  AIRNET  Design  Shortcoming  on  Two  or  More 
Iterations 


Collective  task/subtask  name 

%SMEsa 

%Crewsa 

Move  to  Holding  Area13 

Depart  current  location  on  time 

43 

43 

Conduct  Lines  of  Passage13 

Comply  with  time  windows 

0 

43 

Arrive  at  Holding  Area13 

Maintain  nap-of-the  earth  flight 

0 

29 

Establish  ha3ty  security 

29 

0 

Conduct  Tactical  Air  Movement13 

Depart  on  time 

14 

14 

Recon  enemy  avenues  of  approach 

0 

29 

Employ  remote  HELLFIRE  designation 

14 

14 

Recon  Battle  Position0 

Recon  attack  routes  into  the  battle  position 

0 

33 

Recon  battle  position;  establish  contact  with 

0 

67 

opposing  forces 

Ensure  maximum  range  shots  for  attack  assets 

0 

67 

Utilize  Passive  Air  Defense  Artillery  Measures13 
Visually  identify  threat  aircraft 

14 

14 

Occupy  Battle  Position13 

Scout:  perform  security;  attack:  move  into 

14 

14 

battle  position 

Scout:  designate  targets  for  attack  assets; 

14 

29 

attack:  establish  firing  positions 

Scout/weapons  team  perform  remote  HELLFIRE 

43 

29 

engagement 

Actions  on  Contact,  if  Not  Observed13 

Deploy  to  cover 

0 

29 

Maintain  observation  of  enemy  forces 

0 

29 

Percentage  of  subject  matter  experts  (SMEs)  and  crews  reporting  a 
problem  attributable  to  design  shortcomings. 

^Iterations  =  seven  (3cout  and  attack) . 

iterations  =  three  (scout)  . 

^Iterations  =  four  (attack) . 
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Table  5  (Continued) 


Collective  task/subtask  name 

Actions  on  Contact,  if  Observed0 

Deploy  to  cover  and  employ  suppressive  fire 
Maintain  observation  of  enemy  forces 

Lead  Reacts  to  Opposing  Forces0 

Perform  security  and  acquire  additional  targets 
Maintain  contact  with  the  enemy 

Conduct  Target  Handover0 
Identify  targets 

Handover  target  to  attack  assets  and  release 
engagement  authority 

Continue  identification  and  provide  security 
Maintain  contact  with  enemy 

Downed  Aircrew  Recovery^ 

Move  to  preplanned  pickup  point 
Identify  downed  helicopter 
Estimate  aircraft  damage 
Report  enemy  situation  in  the  area 
Report  accessibility  of  the  aircraft 

Perform  Terrain  Flight  and  Counter  Air  Defense 
Artillery  Measures13 
Perform  terrain  flight 
Expedite  traversing  open  areas 
Mask  aircraft 

React  to  Initial  Contact  With  Opposing  Forces13 

Identify  force  a3  enemy  and  report  the  situation 

Engage  Targets13 

Employ  remote  HELLFIRE  with  scout  aircraft 

Deliberate  Air  Combat*-1 

Maneuver  opposing  air  threat  to  disadvantage 


%SMEsa 

%Crewsa 

33 

33 

33 

33 

0 

67 

33 

33 

33 

67 

33 

33 

33 

67 

33 

67 

0 

50 

0 

50 

0 

50 

0 

50 

0 

50 

14 

14 

29 

14 

14 

14 

29 

0 

57 

29 

50 

25 

Percentage  of  subject  matter  experts  (SMEs)  and  crews  reporting  a 
problem  attributable  to  design  shortcomings. 

iterations  =  seven  (scout  and  attack)  . 


iterations  =  three  (scout)  . 
iterations  =  four  (attack)  . 


AIRNET  Component  Design  Shortcomings 


During  the  evaluation  sessions,  the  SMEs  were  instructed 
to  record  their  judgments  about  contributing  AIRNET  design 
shortcomings  each  time  the  performance  of  a  collective 
subtask  was  judged  to  be  inadequate.  Probably  because  of 
time  pressure,  SMEs  seldom  recorded  contributing  design 
shortcomings  on  the  data  sheet.  A  tabulation  of  the  few 
judgments  recorded  on  the  data  sheets  revealed  the  following 
eight  design  shortcomings.  The  numbers  in  parentheses  show 
the  frequency  with  which  each  shortcoming  was  identified. 

•  Difficult  to  control  aircraft  during  terrain  flight  (9) 

•  No  remote  designation  capability  (9) 

•  Difficult  to  make  aircraft  decelerate  (8) 

•  Limited  detection/identification  range  (7) 

•  Limited  lateral  FOV  (7) 

•  Difficult  to  control  aircraft  during  firing  (4) 

•  Topography  unrealistically  sparse  (4) 

•  Radio  equipment  difficult  to  operate  (4) 

Although  these  data  are  suggestive,  they  are  not  a 
reliable  indication  of  the  the  types  of  design  shortcomings 
that  contribute  to  inadequate  performance  of  collective  tasks 
or  the  frequency  with  which  such  design  shortcomings  degrade 
performance.  It  was  for  this  reason  that  two  of  the  SMEs 
were  recalled  after  the  completion  of  the  evaluation  sessions 
to  perform  a  post  hoc  assessment.  It  will  be  recalled  from 
the  Methods  section  of  this  report  that  two  SMEs  (a) 
considered  each  collective  task  for  which  performance  was 
judged  inadequate  and  (b)  identified  the  AIRNET  design 
shortcomings  that,  in  their  judgment,  contributed  to  the 
difficulty  in  performing  the  task.  The  results  of  the  post 
hoc  assessment  are  presented  below. 

In  all,  80  shortcomings  were  identified  for  the  35 
problem  subtasks  (performance  inadequate  on  at  least  two 
iterations,  and  inadequate  performance  attributed  to  AIRNET 
shortcoming) .  The  number  of  shortcomings  identified  per 
subtask  varied  from  1  to  4 .  The  80  design  shortcomings  can 
be  classified  into  13  types.  Four  types  were  associated  with 
temporary  reliability  or  maintenance  problems  that  do  not 
constitute  a  fundamental  design  problem;  these  shortcomings 
are  not  discussed  further.  Table  6  shows  the  type  and 
relative  frequency  of  the  shortcomings  that  are  not 
associated  with  temporary  reliability  or  maintenance 
problems . 

The  types  of  shortcomings  identified  during  the  post  hoc 
assessment  are  similar,  but  not  identical,  to  the  types  of 
shortcomings  identified  during  the  performance  evaluation 
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Table  6 


Type  and  Relative  Frequency  of  AIRNET  Component  Design 
Shortcomings  Identified  for  35  Problem  Collective  Subtasks 


Percentage 


AIRNET  component  design  shortcoming  of  tasks® 

Limited  detection/identification  range  45.71 
Limited  lateral  field  of  view  31.43 
Difficult  to  control  aircraft  during  terrain  flight  25  71 

Difficult  to  make  aircraft  decelerate  20.00 
Topography  unrealistically  sparse  11.43 
No  remote  designation  capability  8.57 
Search  and  rescue  operations  not  realistic  5.71 
Difficult  to  control  aircraft  during  firing  2.86 
No  automatic  target  handover  system  2.86 


®Values  represent  the  percentage  of  problem  collective 
subtasks  (n  =  35)  for  which  associated  shortcoming  was  judged 
to  have  contributed  to  inadequate  performance. 


sessions  (see  list  on  page  30) .  Three  shortcomings  were  not 
common  to  both  lists:  radio  equipment  difficult  to  operate 
(evaluation  sessions  only) ,  search  and  rescue  (SAR) 
operations  not  realistic  (post  hoc  assessment  only) ,  and  no 
automatic  target  handover  system  (ATHS)  (post  hoc  assessment 
only) .  There  is  only  a  moderate  degree  of  commonality  in  the 
frequency  with  which  the  design  shortcomings  were  identified 
during  the  performance  evaluation  sessions  and  the  post  hoc 
assessment.  Some  of  the  differences  stem  from  the  fact  that 
the  data  from  the  performance  evaluation  sessions  include  all 
subtasks,  whereas  the  data  from  the  post  hoc  assessment 
include  only  the  35  problem  subtasks. 

Although  all  the  data  should  be  taken  into  account  in 
considering  desirable  AIRNET  improvements,  the  authors 
believe  that  the  data  compiled  during  the  post  hoc  assessment 
(Table  6)  provide  the  best  estimate  of  the  importance  of 
AIRNET  design  shortcomings.  The  best  index  of  the  importance 
of  a  shortcoming  is  the  percentage  of  tasks  that  were 
adversely  influenced  by  the  design  shortcoming.  Such 
percentage  values  are  shown  in  the  right-hand  column  of 
Table  6 . 
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Many  tasks  are  adversely  influenced  by  shortcomings  in 
the  AIRNET  visual  system.  These  data  clearly  indicate  that 
the  two  most  important  shortcomings  are  limited  detection  and 
identification  range  and  limited  lateral  FOV.  As  would  be 
expected,  such  shortcomings  degrade  collective  tasks  that 
require  crewmembers  to  search  for,  detect,  and  evaluate 
objects  at  representative  standoff  ranges.  Listed  below  are 
subtasks  for  which  performance  was  degraded  by  limited  FOV, 
limited  viewing  range,  or  both: 

•  establish  hasty  security  (FOV) , 

•  recon  enemy  avenues  of  approach  (FOV) , 

•  recon  attack  routes  into  the  battle  position  (FOV) , 

•  recon  battle  position  and  establish  contact  with 
opposing  forces  (both) , 

•  ensure  maximum  range  shots  for  attack  assets  (range) , 

•  visually  identify  threat  aircraft  (range) , 

•  perform  security  and  move  into  battle  position 
(range) , 

•  establish  firing  positions  (range) , 

•  maintain  observation  of  enemy  forces  (range) , 

•  perform  security  and  acquire  additional  targets 
(both) , 

•  maintain  contact  with  the  enemy  (range) , 

•  identify  targets  (range) , 

•  handover  target  to  attack  assets  and  release  (range) , 

•  continue  identification  and  provide  security  (both) , 

•  continue  contact  with  enemy  (both) , 

•  identify  downed  helicopter  (both) , 

•  report  enemy  situation  in  area  (both) , 

•  identify  force  as  enemy  and  report  the  situation 
(both)  ,  and 

•  maneuver  opposing  air  threat  to  disadvantage  (both) . 

Two  other  visual  system  shortcomings  are  associated  with 
the  realism  of  features  portrayed  by  the  computer  generated 
display  system.  One  shortcoming  is  that  topography  is 
unrealistically  sparse;  SMEs  identified  this  shortcoming  as  a 
contributor  to  inadequate  performance  for  about  11%  of  the 
problem  subtasks.  e  subtasks  degraded  by  sparse  topography 
are  ones  in  which  wmembers  must  deploy  to  cover  or  mask 
their  aircraft  during  an  attack  on  targets.  The  other 
shortcoming  is  the  lack  of  realism  of  SAR  operations;  this 
shortcoming  contributed  to  inadequate  performance  of  only  one 
subtask.  It  is  of  interest  to  note  that  there  is  no 
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indication  that  unrealistic  display  features  contributed  to 
spatial  disorientation  or  to  errors  in  distance  or  velocity 
judgments . 

The  second  most  important  class  of  shortcomings  is 
associated  with  difficulties  in  controlling  the  simulated 
aircraft  in  certain  circumstances.  Shortcomings  of  this 
class  include:  (a)  difficult  to  control  aircraft  during 
terrain  flight  (about  26%  of  subtasks),  (b)  difficult  to  make 
aircraft  decelerate  (20%  of  subtasks) ,  and  (c)  difficult  to 
control  aircraft  during  firing  (about  6%  of  subtasks) . 
Together,  these  three  shortcomings  contributed  to  the 
inadequate  performance  of  about  31%  of  the  problem  subtasks. 
The  subtasks  that  are  degraded  tend  to  be  among  the  most 
important  ones.  Difficulty  in  controlling  the  simulated 
aircraft  at  low  altitudes  degrade  NOE  navigation,  deployment 
to  cover,  masking,  and  firing.  Even  if  the  pilot  is  able  to 
maneuver  the  aircraft  in  these  situations,  it  is  likely  that 
the  pilot's  attention  will  be  distracted  from  tactical  tasks 
or  that  high  workload  will  prevent  the  performance  of  the 
tactical  tasks.  Furthermore,  difficulties  in  controlling  the 
simulated  aircraft  are  certain  to  lead  to  crashes,  which 
interrupt  training  for  the  entire  team. 

The  remaining  two  shortcomings  listed  in  Table  6  are 
associated  with  aircraft  capabilities  that  AIRNET  does  not 
have.  Performance  of  some  subtasks  was  not  possible  because 
AIRNET  provides  no  remote  designation  capability  and  has  no 
ATHS .  The  lack  of  these  two  capabilities  degraded 
performance  on  about  11%  of  the  35  problem  subtasks. 

Although  i  relatively  small  percentage  of  subtasks  is 
degraded  by  the  lack  of  a  remote  designation  and  ATHS 
capability,  they  are  tasks  for  which  success  is  heavily 
dependent  upon  precise  team  coordination  and,  therefore,  are 
ideally  suited  for  a  team  training  device  such  as  AIRNET. 

In  the  post  hoc  assessment,  SMEs  did  not  identify  any 
subtasks  for  which  inadequate  performance  was  due  to 
shortcomings  in  communication  system.  However,  spontaneous 
reports  by  crewmembers  indicate  that  the  communication  system 
controls  are  difficult  to  operate  and  that  there  are  too  few 
communication  channels.  These  spontaneous  reports  are 
supported  by  the  questionnaire  data  presented  in  the  next 
section . 


The  data  reported  in  this  section  of  the  report  are 
tabulations  of  participants'  responses  to  items  that  appear 
in  one  or  both  of  the  questionnaires.  It  will  be  recalled 
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from  the  Methods  section  that  both  SMEs  and  crewmembers 
completed  Questionnaire  #1  (Ql,  Appendix  C)  on  two  occasions — 
once  before  the  performance  assessment  sessions  and  again 
immediately  after  completing  the  sessions.  Results  labeled 
initial  refer  to  responses  on  the  first  administration  of  Ql; 
results  labeled  final  refer  to  responses  on  the  second 
administration  of  Ql .  A  comparison  of  the  initial  and  final 
responses  to  the  same  item  provides  an  indication  of  the 
extent  to  which  the  experience  gained  during  the  performance 
evaluation  sessions  influenced  participants'  judgments  about 
the  adequacy  of  AIRNET  components. 

In  addition,  participants  completed  Questionnaire  #2 
(Q2,  Appendix  D)  immediately  after  completing  the  collective 
task  assessment  sessions.  Some  items  on  Q2  were  the  same  as 
items  on  Ql .  For  items  common  to  both  Ql  and  Q2,  only  the 
initial  and  final  responses  on  Ql  are  reported.  The  Q2 
responses  to  common  items  were  so  similar  to  the  final  Ql 
responses  that  nothing  is  gained  by  presenting  and  discussing 
Q2  responses. 

Most  questionnaire  items  required  the  participant  to 
rate  the  adequacy  of  an  AIRNET  system  component  or  feature 
for  performing  each  of  a  set  of  prescribed  tasks.  A  5-point 
rating  scale  was  used  in  all  cases.  The  rating  values  and 
verbal  anchors  are  as  follows: 

•  1  =  totally  inadequate, 

•  2  =  somewhat  inadequate, 

•  3  =  just  adequate  enough, 

•  4  =  more  than  adequate,  and 

•  5  =  much  more  than  adequate. 

After  examining  the  results,  the  authors  judged  that 
presenting  and  discussing  the  percentage  of  participants  who 
selected  each  of  the  five  rating  values  would  add  little 
useful  information.  Hence,  the  results  are  presented  simply 
as  the  percentage  of  respondents  who  judged  the  AIRNET 
component  or  feature  in  question  to  be  inadequate  (rating 
values  1  or  2)  or  adequate  (rating  values  3,  4,  or  5) .  For 
example,  if  5%  of  the  participants  selected  a  rating  value  of 
1  (totally  inadequate)  and  20%  selected  a  rating  value  of  2 
(somewhat  inadequate) ,  the  results  are  described  by  stating 
that  25%  of  participants  rated  the  component  inadequate. 

The  following  subsections  describe  participants' 
judgments  of  the  adequacy  of  the  visual  system,  the  flight 
characteristics,  the  crew  stations,  and  the  communication 
equipment.  Also,  data  are  presented  on  participants' 
judgments  about  the  maximum  target  acquisition  range  (unaided 
and  FLIR)  and  the  probable  training  value  of  AIRNET. 
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Judged  Adequacy  of  Visual  System  Attributes 


Because  the  visual  system  is  certain  to  influence  the 
training  effectiveness  of  any  simulation  system,  both  Q1  and 
Q2  contain  items  that  required  participants  to  judge  the 
adequacy  of  the  visual  system  in  general  and  the  adequacy  of 
specific  attributes  of  the  visual  system.  Because  the 
adequacy  of  a  visual  system  is  at  least  partially  task 
dependent,  every  visual  system  item  on  the  questionnaires 
required  participants  to  evaluate  the  adequacy  of  the  visual 
system  for  performing  each  of  a  limited  set  of  prescribed 
tasks . 

Described  below  are  crewmembers'  judgments  about  the 
adequacy  of  the  (a)  lateral  FOV,  (b)  visual  cues  and  visual 
scene  (general),  (c)  stability  and  smoothness  of  the  visual 
scene,  (d)  scene  realism,  and  (e)  discernibility  and  realism 
of  topographic  features .  Also  presented  are  crewmembers' 
judgments  about  the  visual  system' s  propensity  for  creating 
visual  problems. 

Lateral  FOV.  Participants  were  required  to  rate  the 
adequacy  of  lateral  FOV  for  performing  each  of  13  tasks  (see 
Ql,  item  1) .  The  initial  and  final  ratings  are  summarized  in 
Figure  7 .  The  values  represent  the  percentage  of 
participants  who  judged  lateral  FOV  to  be  inadequate  for 
performing  the  associated  task.  The  tasks  are  ordered  in 
terms  of  the  percentage  judged  inadequate  on  the  final 
rating;  the  task  with  the  largest  percentage  is  listed  first. 

For  nine  tasks,  more  participants  judged  lateral  FOV 
inadequate  on  the  final  rating  than  on  the  initial  rating. 
Hovering,  situation  awareness,  and  normal  takeoff  and  landing 
are  the  only  tasks  for  which  more  participants  judged  lateral 
FOV  to  be  inadequate  on  the  initial  than  on  the  final  rating. 
The  results  suggest  that  the  experience  gained  during  the 
collective  task  evaluation  sessions  gave  participants  a 
better  understanding  of  the  extent  to  which  a  limited  lateral 
FOV  degrades  performance  on  some  tasks.  However,  the 
difference  was  large  enough  to  reach  statistical  significance 
for  only  one  task,  formation  flight.  Because  the  final 
judgments  are  based  upon  more  experience  than  the  initial 
judgments,  the  discussion  of  ratings  by  task  is  limited  to 
judgments  of  lateral  FOV  adequacy  on  the  final  rating. 

The  results  indicate  that  lateral  FOV  is  judged  most 
inadequate  for  formation  flight,  deceleration,  threat 
detection  and  avoidance,  and  tasks  that  require  flying  very 
low  or  very  fast.  Approximately  81%  of  the  participants 
judged  lateral  FOV  inadequate  for  formation  flight,  and 
approximately  67%  judged  lateral  FOV  inadequate  for 
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TASK  NAME 
Formation  Flight 

Deceleration 

Threat  Detection/Avoidance 

Forward  Flight  100+  Knots 

Nap-of-the-Earth  Flight 

Forward  Flight  50-100  Knots 

Hovering 

Situation  Awareness 

Contour  Flight 

Target  Acquisition 

Forward  Flight  0-50  Knots 

Low  Level  Flight 

Normal  Takeoff /Landing 

0  20  40  60  80 

Percentage  of  Participants 
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Initial  Rating 
Final  Rating 
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aH  =  7;  «  9;  cn  —  10;  otherwise  a  =  12  for  initial  rating  and  a  —  11 

for  final  rating. 

Figure  7.  Percentage  of  participants  who  judged  the  field  of 
view  inadequate  during  the  initial  and  final  ratings. 


deceleration.  About  55%  of  the  participants  judged  lateral 
FOV  inadequate  for  performing  threat  detection  and  avoidance. 
Between  45%  and  55%  of  the  participants  judged  lateral  FOV 
inadequate  for  forward  flight  at  speeds  greater  than  50  knots 
and  for  NOE  flight.  No  more  than  36%  judged  lateral  FOV 
inadequate  for  the  remaining  seven  tasks . 

Although  not  shown  in  Figure  7,  there  was  no  task  for 
which  more  than  20%  of  the  participants  rated  lateral  FOV 
higher  than  3  (just  adequate  enough)  on  the  final  rating; 
only  16%  of  all  final  ratings  were  larger  than  3.  Hence, 
even  when  a  relatively  large  percentage  of  participants 
judged  lateral  FOV  to  be  adequate  for  performing  a  task,  they 
seldom  judged  it  to  be  more  than  marginally  adequate. 
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Visual  cues  and  visual  scene  (general) .  Item  2  on  Q1 
required  participants  to  assess  the  adequacy  of  the  visual 
cues  provided  by  the  extra-cockpit  display  for  nine  tasks. 

The  results  of  the  initial  and  final  ratings  of  visual  cue 
adequacy  are  summarized  in  Figure  8.  For  all  tasks  except 
three  (hovering,  contour  flight,  and  low  level  flight) ,  a 
larger  percentage  of  participants  judged  visual  cues  to  be 
inadequate  on  the  initial  than  on  the  final  rating.  However, 
because  none  of  the  differences  between  initial  and  final 
ratings  were  statistically  significant,  it  cannot  be 
concluded  that  the  judged  adequacy  of  visual  cues  increases 
with  experience  in  AIRNET. 

An  examination  of  the  final  judgments  shows  that  there 
are  only  three  tasks  for  which  one-third  or  more  of  the 
participants  judged  visual  cues  to  be  inadequate:  58%  for 
hovering,  55%  for  deceleration,  and  33%  for  contour  flight. 
For  the  remaining  six  tasks,  no  more  than  25%  of  the 
participants  judged  visual  cues  to  be  inadequate.  As  was 


TASK  NAME 
Hovering  Flight 

Deceleration 

Contour  Flight 

Forward  Flight  100+  Knots 

Nap-of-the-Earth  Flight 

Low  Level  Flight 

Normal  Takeoff /Landing 

Forward  Flight  50-100  Knots 

Forward  Flight  0-50  Knots 


0  20  40  60  80 

Percentage  of  Participants 

an  =  11;  all  other  ns  =  12. 

Figure  8.  Percentage  of  participants  who  judged  the  visual 
cues  in  the  scene  inadequate  during  the  initial  and  final 
ratings . 
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true  for  lateral  FOV,  the  judged  inadequacy  of  visual  cues 
for  performing  decelerations  may  be  inflated  by  the 
difficulty  encountered  in  performing  decelerations  in  AIRNET. 

Item  5  on  Q1  required  participants  to  rate  the  adequacy 
of  the  visual  scene  for  performing  tasks  during  a  gunnery 
operation.  About  75%  of  the  participants  judged  the  visual 
scene  to  be  inadequate  for  situation  awareness  and  target 
acquisition.  For  all  other  tasks,  however,  no  more  than  25% 
of  the  participants  judged  the  visual  scene  to  be  inadequate. 

The  similarity  of  the  responses  to  the  items  on  visual 
cues  (general)  and  visual  scene  (general)  suggests  that 
participants  were  responding  to  much  the  same  attributes  when 
responding  to  the  two  items.  When  considered  together,  the 
responses  to  the  two  items  indicate  that  a  clear  majority  of 
participants  consider  the  visual  scene  and  the  cues  therein 
to  be  inadequate  for  situation  awareness,  target  acquisition, 
hovering,  and  deceleration.  For  all  other  tasks  assessed, 
the  visual  scene  and  cues  were  judged  inadequate  by  one-third 
or  fewer  participants. 

Stability  and  smoothness  of  visual  scene.  Participants 
rated  the  adequacy  of  the  stability  and  smoothness  of  the 
visual  scene  for  performing  four  tasks.  Stability  was  rated 
only  on  Q1  (Item  3)  and  smoothness  was  rated  only  on  Q2  (Item 
4) .  The  data  show  that  the  ratings  were  highly  similar  for 
stability  and  smoothness  and  were  highly  consistent  across 
tasks.  There  was  no  task  for  which  more  than  25%  of  the 
participants  judged  stability  to  be  inadequate;  the  range  of 
percentage  values  varied  from  17%  to  25%.  Similarly,  there 
was  no  task  for  which  more  than  about  30%  of  participants 
judged  smoothness  to  be  inadequate.  These  data  provide  no 
indication  that  either  the  stability  or  the  smoothness  of  the 
AIRNET  extra-cockpit  display  seriously  degrades  performance 
on  any  task  investigated. 

Realism  of  visual  scene.  Item  3  on  Q2  required 
participants  to  rate  the  adequacy  of  the  visual  scene  realism 
for  four  tasks:  altitude  detection,  adjusting  artillery 
fire,  calling  for  artillery  fire,  and  navigation.  The  item 
did  not  require  participants  to  assess  visual  scene  realism 
with  respect  to  the  real  world;  rather,  they  were  asked  to 
indicate  whether  or  not  the  scene  was  sufficiently  realistic 
to  perform  prescribed  tasks. 

The  data  show  that  more  than  one-half  of  the 
participants  judged  scene  realism  to  be  inadequate  for  three 
tasks:  altitude  detection  (67%),  adjusting  artillery  fire 
(60%),  and  calling  for  artillery  fire  (53%).  In  contrast, 
only  27%  of  the  participants  judged  visual  scene  realism  to 
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be  inadequate  for  navigation.  Although  the  AIRNET  visual 
scene  is  not  realistic  in  an  absolute  sense,  most  of  the 
participants  judged  it  to  be  adequate  for  the  navigation  that 
must  be  accomplished  during  a  team  training  scenario.  These 
data  suggest  that  increased  scene  realism  would  improve 
performance  on  three  of  the  four  tasks  included  on  the 
questionnaire.  Additional  research  is  required  to  specify 
the  scene  elements  that  should  be  modified  and  the  manner  in 
which  they  should  be  modified  to  achieve  sufficient  scene 
realism. 

Discernibility  and  realism  of  topographic  features. 
Participants  were  required  to  rate  the  discernibility  of  the 
topographic  features  portrayed  on  the  extra-cockpit  display 
(Ql,  Item  4).  Participants  were  told  that  the  term 
discernibility  refers  to  the  ease  with  which  a  feature  can  be 
recognized  and  differentiated  from  other  features  when  it  is 
clearly  visible.  The  results  of  the  ratings  of 
discernibility  are  shown  in  Figure  9.  Examination  of  the 
final  ratings  shows  that  there  are  only  three  tasks  for  which 
more  than  40%  of  the  participants  judged  AIRNET  topographic 
features  to  be  inadequately  discernible:  target  acquisition 
(50%),  situation  awareness  (42%),  and  NOE  flight  50  -  100 
knots  (42%) .  For  all  the  other  tasks,  no  more  than  25%  of 


TASK  NAME 

Target  Acquisition 

Situation  Awareness 
Nap-of-the-Earth  Flight  50-100  Knots 
Contour  Flight  50-100  Knots 
Low  Level  Flight  50-100  Knots 
Nap-of-the-Earth  Flight  0-50  Knots 
Contour  Flight  0-50  Knots 
Low  Level  Flight  0-50  Knots 

0  10  20  30  40  50 

Percentage  of  Participants 

aH  =  6;  =  7;  an  other  ns  =  12. 


Fit  i  i — i — i — i — i — T 


gj  Initial 
B  Final 


Figure  9.  Percentage  of  participants  who  judged  the 
topographic  features  to  be  inadequately  discernible  during 
the  initial  and  final  ratings. 


the  participants  judged  topographic  feature  discernibility  to 
be  inadequate  on  the  final  rating.  Although  there  are  some 
relatively  large  differences  between  initial  and  final 
ratings,  none  of  the  differences  are  statistically 
significant . 

For  tasks  common  to  both  items,  the  ratings  of 
topographic  feature  realism  (Q2,  Item  5)  are  nearly  identical 
to  the  ratings  of  visual  scene  realism  (Ql,  Item  4) .  A 
relatively  high  percentage  of  participants  judged  both  scene 
and  topographic  feature  realism  inadequate  for  altitude 
detection  (53%  scene  and  64%  topographic  feature) . 

Conversely,  a  relatively  low  percentage  judged  scene  and 
topographic  feature  realism  inadequate  for  navigation  (21% 
scene  and  27%  topographic  feature) .  Other  tasks  for  which 
the  adequacy  of  topographic  feature  realism  was  rated  include 
NOE  flight  and  masking/unmasking.  The  percentage  of 
participants  who  judged  topographic  feature  realism  to  be 
inadequate  for  NOE  flight  and  masking/unmasking  are  43%  and 
35%,  respectively. 

Vision  problems.  Item  20  on  Q2  required  crewmembers  to 
rate  the  adequacy  of  the  visual  system  with  respect  to  its 
propensity  for  producing  eye  fatigue,  eye  strain,  and  blurred 
vision.  One-half  the  crewmembers  rated  the  visual  system 
inadequate  with  respect  to  eye  strain;  43%  rated  the  visual 
system  inadequate  with  respect  to  eye  fatigue.  Only  29% 
rated  the  visual  system  inadequate  with  respect  to  blurred 
vision . 


Judged  Adequacy  of  Flight  Characteristics 

Three  questionnaire  items  were  designed  to  assess 
crewmembers'  judgments  about  the  adequacy  of  the  simulated 
aircraft's  flight  characteristics.  Item  8  on  Ql  asked 
crewmembers  to  judge  whether  the  simulated  aircraft  flies 
enough  like  a  helicopter  to  enable  them  to  perform  their 
mission.  All  crewmembers  rated  the  aircraft  simulation 
inadequate  on  the  initial  rating;  89%  rated  the  simulation 
inadequate  on  the  final  rating.  Item  9  on  Ql  asked 
crewmembers  to  judge  the  extent  to  which  the  simulated 
aircraft  imposes  realistic  attentional  demands  on  the  pilot. 
The  simulated  aircraft  was  judged  inadequate  with  respect  to 
attentional  demands  by  90%  of  the  crewmembers  on  the  initial 
rating  and  71%  of  the  crewmembers  on  the  final  rating. 

Item  1  on  Q2  asked  crewmembers  to  rate  the  realism  of 
AIRNET's  response  to  a  variety  of  control  inputs; 
crewmembers'  responses  to  this  item  are  summarized  in  Figure 
10.  The  horizontal  bars  in  Figure  10  show  the  percentage  of 
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CONTROL  INPUT 
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Doppler  Correction 
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Figure  10.  Percentage  of  participants  who  judged  the  realism 
of  AIRNET' s  response  to  control  inputs  to  be  inadequate. 


crewmembers  who  judged  the  associated  control  response  to  be 
inadequate.  Examine  first  the  crewmembers'  judgments  about 
the  three  primary  flight  controls:  the  collective,  cyclic, 
and  pedals.  Approximately  71%  of  the  crewmembers  judged  the 
response  to  collective  inputs  to  be  inadequate  and  about  43% 
judged  the  response  to  cyclic  and  pedal  inputs  to  be 
inadequate.  These  ratings  provide  further  support  for  the 
conclusion  that  the  AIRNET  aircraft  simulation  lacks 
sufficient  realism. 

Other  control  inputs  judged  inadequate  by  more  than  40% 
of  the  crewmembers  include  TADS  slewing  (75%) ,  weapon 
selection  (63%),  30  mm  cannon  firing  (56%),  and  Doppler  entry 
(44%) .  Performance  on  tasks  that  require  the  use  of  these 
controls  is  probably  degraded  by  the  lack  of  realism  in 
AIRNET' s  response  to  control  inputs.  The  remaining  control 
inputs  listed  in  Figure  10  were  judged  inadequate  by  one- 
third  or  fewer  of  the  participants,  so  they  are  not 
considered  to  be  a  serious  problem. 
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Judged  Comfort  of  the  Crew  Station 


Crewmembers  were  asked  to  rate  the  comfort  of  prescribed 
attributes  of  the  crew  station  (Item  19,  Q2) .  Only  three 
attributes  were  judged  inadequate  by  more  than  one-third  of 
the  crewmembers:  height  of  the  visual  screens  (50%), 
distance  of  screens  from  instruments  (43%),  and  location  of 
the  instrument  displays  (43%)  .  About  one-third  of  the 
crewmembers  judged  the  comfort  of  the  seats  to  be  inadequate 
for  training  sessions  of  2  hours  or  more.  The  distance  of 
the  screens  from  the  pilot  was  judged  inadequate  (excessive) 
by  only  30%  of  the  crewmembers.  The  location  of  the 
collective  and  the  orientation  of  the  pedals  were  judged 
inadequate  by  fewer  than  10%  of  the  crewmembers. 

Although  these  results  indicate  that  selective  cockpit 
design  modifications  might  improve  user  comfort,  it  cannot  be 
inferred  from  these  data  that  such  improvements  would  result 
in  measurable  improvements  in  AIRNET' s  training 
effectiveness . 


Adequacy  of  Communication  System 

Items  on  Q1  asked  crewmembers  to  rate  the  adequacy  of 
the  AIRNET  communication  system  for  information  transmission 
(Item  11),  information  reception  (Item  12),  ai.'*  monitoring 
two  channels  simultaneously  (Item  13) .  The  ratings  on  all 
three  items  were  highly  similar  on  both  the  initial  and  the 
final  rating.  On  the  final  rating,  between  42%  and  58%  of 
the  crewmembers  judged  the  communication  system  to  be 
inadequate.  The  ratings  of  the  communication  system  did  not 
differ  significantly  as  a  function  of  type  communication 
(transmission  vs  reception)  or  communication  function  (other 
crewmember,  other  aircraft,  or  TOC  commander) .  Discussions 
with  crewmembers  indicated  that  the  main  reason  for  the 
inadequate  rating  of  the  communication  system  is  that  it 
provides  too  few  communication  channels. 


Judged  Maximum  Target  Acquisition  Range 

Items  on  Q1  asked  crewmembers  to  judge  the  maximum  range 
at  which  the  AIRNET  visual  system  supports  unaided  target 
acquisition  (Item  6)  and  target  acquisition  with  FLIR  (Item 
7) .  On  the  final  administration  of  Ql,  crewmembers' 
judgments  corresponded  closely  with  their  performance  on  the 
target  detection  performance  assessment.  For  unaided 
viewing,  83%  of  the  crewmembers  indicated  that  the  maximum 
target  acquisition  range  is  1,000  m  or  less;  only  one 
crewmember  indicated  that  the  maximum  target  acquisition 
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range  exceeds  1,500  m.  For  FLIR  viewing,  75%  of  the 
crewmembers  indicated  that  the  maximum  target  acquisition 
range  is  4,000  m  or  less;  no  one  indicated  that  the  maximum 
target  acquisition  range  with  FLIR  is  greater  than  5,000  m. 


Judged  Training  Value  of  AIRNET 

All  participants  used  a  5-point  rating  scale  to  rate  the 
value  of  AIRNET,  in  its  current  configuration,  for  training 
each  of  25  tasks  (Item  24,  Q2) .  The  rating  values  selected 
for  each  task  were  summed  across  the  15  raters,  and  the 
resulting  values  were  used  to  rank  order  the  tasks.  Table  7 
shows  the  rank-ordered  listing  of  tasks  along  with  the 
ranking  for  each  task.  The  tasks  with  the  highest  ranking 
(smallest  rank  number)  are  the  ones  for  which  AIRNET  was 
judged  to  have  the  greatest  training  value. 

As  would  be  expected,  AIRNET' s  training  value  tends  to 
be  rated  highest  for  collective  tasks  that  require  close 
coordination  between  two  aircraft  crews  or  between  an 
aircraft  crew  and  the  crew  of  a  ground  unit.  For  example, 
collective  tasks  for  which  AIRNET' s  training  value  is  rated 
relatively  high  (among  the  10  highest  ranked  tasks)  include 
close  air  support,  tactics  planning  and  briefing,  call  for 
artillery  fire,  adjusting  artillery  fire,  target  handover, 
low  altitude  tactics,  high  threat  tactics,  and  low  threat 
tactics . 

It  seems  probable  that  limitations  of  the  visual  system 
contributed  to  the  low  ranking  of  some  tasks.  For  instance, 
a  limited  FOV  may  have  contributed  to  the  low  ranking  for 
tasks  such  as  two-ship  tactics,  four-ship  tactics,  and 
tactical  formation.  Similarly,  the  limited  target  detection 
and  identification  range  may  have  contributed  to  the  low 
ranking  for  such  tasks  as  gun  employment,  target  engagement, 
and  visual  identification. 

Although  the  data  in  Table  7  are  useful  for  assessing 
the  relative  value  of  AIRNET  for  training  different  tasks, 
the  reader  is  cautioned  against  assuming  that  such  data  are  a 
reliable  indicator  of  the  absolute  training  value  of  AIRNET. 


43 


Table  7 


Training  Tasks  Rank  Ordered  by 
(N  =  25) 

Rated  Training  Value 

of  AIRNET 

Task  name 

Sum  of 
ratings 

Rank 

Close  air  support 

41 

1 

Tactics  planning  and  briefing 

40 

2 

Call  for  artillery  fire 

38 

3 

Adjusting  artillery  fire 

37 

4.5 

Target  handover 

37 

4.5 

Low  altitude  tactics 

36 

6 

High  threat  tactics 

35 

7 

Low  level  flight 

34 

8.5 

Low  threat  tactics 

34 

8.5 

Safe  passage 

33 

10 

Medium/high  altitude  tactics 

31 

12.5 

Movement  to  contact 

31 

12.5 

Screening  operations 

31 

12.5 

Tactical  intercept 

31 

12.5 

Egress  tactics 

30 

16.5 

Missile  employment 

30 

16.5 

Reconnaissance  missions 

30 

16.5 

Two-ship  tactics 

30 

16.5 

Four-ship  tactics 

29 

20 

Gun  employment 

29 

20 

Nap-of-the-earth  flight 

29 

20 

Communication  procedure 

27 

22 

Target  engagement 

26 

23 

Tactical  formation 

24 

24.5 

Visual  identification 

24 

24.5 
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Discussion  and  Conclusions 


This  research  was  designed  principally  to  assess 
experienced  crewmembers'  ability  to  perform  individual  and 
collective  tasks  in  AIRNET.  The  fundamental  premise 
underlying  the  research  is  that  useful,  albeit  limited, 
inferences  about  AIRNET' s  training  value  can  be  drawn  from 
data  on  experienced  aviators'  ability  to  perform  selected 
tasks  in  AIRNET.  Specifically,  it  is  assumed  that  if  a  task 
that  an  experienced  aviator  performs  routinely  in  the 
aircraft  cannot  be  performed  adequate_y  by  the  aviator  in 
AIRNET,  it  is  unlikely  that  effective  training  on  that  task 
can  be  accomplished  in  AIRNET.  The  research  was  also 
designed  to  identify  AIRNET  design  attributes  that  contribute 
to  crewmembers'  inability  to  perform  key  tasks  in  the  device. 
Finally,  the  research  was  designed  to  assess  participants' 
opinions  about  AIRNET' s  effectiveness  for  training  specific 
flying  tasks. 

The  methodology  employed  yielded  two  types  of  data: 
performance  and  opinion.  Crewmembers'  ability  to  perform 
selected  individual  and  collective  tasks  was  assessed  by 
highly  experienced  SMEs  (instructor  pilots) .  The  SMEs 
assessed  crewmembers'  ability  to  perform  13  individual  flying 
tasks  in  AIRNET  and  assessed  the  maximum  range  at  which 
crewmembers  can  detect  and  identify  targets  in  AIRNET. 
Performance  on  collective  tasks  and  subtasks  was  assessed  by 
SMEs  in  the  context  of  tactically  and  doctrinally  valid 
scenarios  for  each  of  three  types  of  missions  for  light- 
attack  and  heavy-attack  helicopter  teams:  a  cross-FLOT 
deliberate  air  attack,  a  deliberate  offensive  attack,  and  a 
hasty  attack. 

Participants'  (crewmembers  and  SMEs)  opinions  were 
assessed  by  means  of  questionnaires  that  were  administered  at 
three  points  during  the  evaluation  period.  The 
questionnaires  included  items  that  required  participants  to 
(a)  rate  the  adequacy  of  specific  AIRNET  components  for 
performing  specified  tasks,  (b)  rate  the  maximum  range  at 
which  targets  can  be  detected  (unaided  and  FLIR) ,  (c) 

identify  the  physiological  discomfort  and  simulator  sickness 
symptoms  experienced  in  AIRNET,  and  (d)  rate  the  probable 
training  effectiveness  of  AIRNET.  In  addition,  participants 
were  requested  to  identify  AIRNET  system  components  that,  in 
their  judgments,  contributed  to  inadequate  performance  of 
collective  tasks.  All  crewmembers  and  SMEs  were  requested  to 
make  such  judgments  during  the  collective  task  assessment 
sessions.  However,  because  participants  often  failed  to 
record  their  judgments  during  the  evaluation  sessions,  two 
SMEs  subsequently  made  judgments  about  AIRNET  components  that 
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contributed  to  inadequate  performance  for  each  of  35 
(collective)  subtasks. 

The  composite  research  findings  are  discussed  below  as 
they  bear  on  the  following  questions. 

•  Of  the  tasks  investigated,  what  tasks  can  and  what 
tasks  cannot  be  performed  adequately  in  AIRNET  by 
experienced  crewmembers? 

•  When  certain  tasks  cannot  be  performed  adequately  in 
AIRNET,  what  inferences  can  be  drawn  from  these 
findings  about  the  probable  training  effectiveness  of 
AIRNET? 

•  What  AIRNET  components  contribute  to  the  inadequate 
performance  of  tasks  and  how  must  these  components  be 
modified  to  provide  for  adequate  performance? 


Individual  Flying  Tasks 

The  results  of  the  assessment  of  individual  flying  task 
performance  indicate  that,  with  sufficient  practice, 
crewmembers  can  adequately  perform  most  of  the  individual 
tasks  investigated,  including  hovering  flight,  accelerations, 
navigation  at  various  speeds,  low  level  flight,  contour 
flight,  and  NOE  flight.  However,  four  tasks  could  not  be 
performed  adequately  even  after  substantial  experience  in 
AIRNET:  vertical  masking,  lateral  masking,  normal 

deceleration,  and  NOE  deceleration.  Inadequate  performance 
on  these  four  tasks,  particularly  decelerations,  is  unlikely 
to  be  overcome  with  a  reasonable  amount  of  practice  in 
AIRNET. 

It  can  be  argued  that,  since  AIRNET  was  not  developed  to 
teach  aviators  to  perform  individual  flying  tasks,  the  ease 
or  difficulty  of  performing  individual  flying  tasks  has 
little  bearing  on  AIRNET' s  effectiveness  for  training 
collective  tasks.  There  are  at  least  two  reasons  why  this 
argument  is  not  valid.  First,  difficulty  in  performing 
individual  flying  tasks  may  seriously  degrade  the  efficiency 
of  training  collective  tasks.  A  crash  during  a  team  training 
exercise  delays  training  for  the  entire  team  and,  more 
importantly,  interferes  with  the  continuity  and  flow  of  the 
exercise.  Even  if  crashes  are  not  frequent,  difficulty  in 
performing  individual  flying  tasks  may  create  unrealistically 
high  workload  that,  in  turn,  prevents  crewmembers  from 
attending  to  the  collective  tasks  for  which  AIRNET  was 
designed  to  train.  Second,  excessive  difficulty  in 
performing  individual  flying  tasks  may  adversely  affect 
trainees'  and  trainers'  judgments  about  AIRNET' s  utility  for 
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training  collective  tasks.  Negative  attitudes  about  a 
device's  training  effectiveness  are  certain  to  have  a 
negative  influence  on  the  frequency  and  effectiveness  with 
which  the  device  is  used. 

These  arguments  are  particularly  germane  for  the 
individual  tasks  that  crewmembers  had  the  most  difficulty 
performing  in  AIRNET.  Masking  (lateral  and  vertical)  and 
decelerations  (normal  and  NOE)  are  tasks  that  must  be 
performed  frequently  in  nearly  any  attack  team  training 
scenario;  furthermore,  they  are  tasks  for  which  crash 
likelihood  is  high.  For  these  reasons,  the  authors  believe 
that  AIRNET' s  training  effectiveness  is  significantly 
degraded  by  the  difficulty  associated  with  performing  masking 
and  deceleration  tasks.  Furthermore,  spontaneous  comments 
and  questionnaire  responses  by  crewmembers  suggest  that 
AIRNET' s  training  effectiveness  also  may  be  degraded  by  the 
excessive  amount  of  effort  that  must  be  expended  to  perform 
some  of  the  individual  flying  tasks  that  crewmembers  can,  in 
fact,  learn  to  perform  to  standards  in  AIRNET.  Crewmember 
comments  and  questionnaire  responses  suggest  that  excessive 
effort  may  be  required  to  perform  hovering,  contour  flight, 
NOE  flight,  and  high  speed  forward  flight  tasks. 

The  evidence  indicates  that  the  difficulty  in  performing 
individual  tasks  stems  mainly  from  the  aircraft  equations  of 
motion  employed  in  AIRNET.  On  one  questionnaire  item  (Ql, 
Item  8) ,  89%  of  the  crewmembers  rated  AIRNET  inadequate  with 
respect  to  the  extent  that  it  flies  like  a  helicopter.  On 
another  item  (Ql,  Item  9) ,  71%  of  the  crewmembers  indicated 
that  AIRNET  imposes  unrealistically  high  attentional  demands 
on  the  person  flying  it.  In  a  third  item  (Q2,  Item  1),  a 
large  proportion  of  the  crewmembers  indicated  that  AIRNET' s 
responses  to  cyclic  inputs,  collective  inputs,  and  pedal 
inputs  are  not  sufficiently  realistic;  71%  judged  collective 
responses  to  be  inadequate  and  43%  judged  cyclic  and  pedal 
responses  to  be  inadequate.  Responses  to  questionnaire  items 
about  the  visual  system  indicate  that  the  limited  FOV  and 
scene  content  contributed  to  difficulty  in  performing  some 
individual  flying  tasks,  especially  decelerations,  hovering, 
NOE  flight,  and  contour  flight. 

The  composite  results  from  the  individual  task 
assessment  support  the  conclusions  that  (a)  AIRNET' s  training 
effectiveness  is  degraded  by  the  difficulty  associated  with 
performing  some  individual  flying  tasks  and  (b)  improved 
equations  of  motion  is  an  essential  first  step  in  improving 
AIRNET' s  performance.  The  results  also  suggest  that 
performance  benefits  may  be  realized  from  a  wider  FOV  and  a 
more  realistic  scene  content. 
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Target  Detection  and  Identification  Range 


Target  detection  and  identification  is  an  essential  part 
of  any  useful  attack  team  training  scenario.  As  a 
consequence,  the  training  effectiveness  of  any  device 
intended  for  attack  team  training  is  certain  to  be  influenced 
by  the  range  at  which  targets  can  be  detected  and  identified. 
The  results  of  the  assessment  of  target  detection  and 
identification  performance  leave  no  doubt  that  the  maximum 
target  detection  and  identification  range  with  unaided  vision 
is  less  in  AIRNET  than  is  possible  in  an  aircraft  under  good 
visibility  conditions  However,  the  key  question  is  whether 
unaided  target  detection  and  identification  range  is  great 
enough  to  support  effective  training. 

The  data  indicate  that,  with  unaided  vision,  reliable 
target  detection  in  AIRNET  cannot  be  expected  at  ranges 
beyond  about  1,500  m.  Even  though  identification  of  a  target 
as  friend  or  foe  in  AIRNET  requires  only  a  color 
discrimination,  maximum  (unaided)  IFF  range  is  less  than 
maximum  target  detection  range.  The  maximum  range  for 
unaided  identification  of  specific  targets  (tank,  truck, 
etc.)  in  AIRNET  is  considerably  less  than  maximum  IFF  range. 

The  maximum  unaided  target  detection  and  identification 
range  in  an  aircraft  varies  as  a  function  of  lighting 
conditions,  atmospheric  attenuation  (dust,  smoke,  or  water 
particles) ,  and  visual  obstructions  (terrain  relief, 
vegetation,  and  cultural  features) .  Hence,  it  is  difficult 
to  specify  a  single  range  to  use  as  a  standard  for  evaluating 
AIRNET.  For  purposes  of  this  discussion,  it  is  assumed  that 
effective  training  would  be  possible  in  AIRNET  if  (a)  unaided 
target  detection  can  be  accomplished  reliably  at  a  range  of 
about  3,500  m,  (b)  unaided  IFF  can  be  accomplished  at  a  range 
of  about  2,500  m,  and  (c)  unaided  identification  of  a 
specific  target  type  can  be  accomplished  at  a  range  of  about 
1,500  m.4  These  ranges  far  exceed  the  maximum  target 
detection  and  identification  (unaided)  ranges  in  AIRNET.  As 
a  consequence,  the  authors  conclude  that  AIRNET' s  training 
value  is  diminished  substantially  by  the  limited  target 
detection  and  identification  range. 

Target  detection  with  the  AIRNET  FLIR  can  be 
accomplished  reliably  at  ranges  to  6,000  m.  IFF 
identification  can  be  accomplished  with  high  (but  not  100%) 


4The  assumptions  about  ranges  to  use  as  standards  for  AIRNET 
are  based  on  information  gained  during  informal  discussions 
with  Army  aviators;  however,  they  do  not  reflect  the 
official  judgment  of  any  Army  agency. 
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reliability  to  a  range  of  about  4,000  m.  The  range  at  which 
specific  targets  can  be  identified  reliably  with  FLIR  is 
less.  Although  all  targets  were  correctly  identified  at  a 
range  of  1,000  m,  only  one-half  of  the  targets  were  correctly 
identified  at  ranges  between  2,000  and  4,000  m,  and  none  were 
correctly  identified  at  ranges  greater  than  4,000  m. 

As  was  true  for  unaided  vision,  the  maximum  target 
detection  and  identification  range  with  an  operational  FLIR 
varies  with  conditions.  Discussions  with  Army  aviators 
indicated  that,  under  near  optimal  conditions  and  with 
magnification,  targets  can  be  detected  to  a  range  of  8,000  m 
and  can  be  identified  (specific  target  type)  at  a  range  of 
6,500  m. 

The  authors  believe  that  target  detection  range  with  the 
AIRNET  FLIR  is  probably  great  enough  to  support  effective 
training.  However,  target  identification  range  (IFF  and 
specific)  with  AIRNET  FLIR  is  probably  not  great  enough  to 
support  effective  training. 

The  research  produced  other  data  that  bear  on  the 
adequacy  of  target  detection  and  identification  ranges  in 
AIRNET.  Inadequate  target  acquisition  range  was  identified 
in  both  the  collective  task  assessment  and  the  post  hoc 
assessment  as  a  factor  that  contributed  to  the  inadequate 
performance  of  collective  tasks.  Inadequate  target 
acquisition  range  was  the  most  frequently  identified  factor 
in  the  post  hoc  assessment  and  the  fourth  most  frequently 
identified  factor  during  the  collective  task  assessment. 
Responses  to  one  questionnaire  item  (Ql,  Item  4)  indicated 
that  50%  of  the  crewmembers  judged  that  the  visual  system's 
discernibility  was  inadequate  to  support  target  acquisition. 
Finally,  crewmembers  judged  the  training  value  of  AIRNET  to 
be  lower  for  visual  identification  than  for  any  other  task 
investigated  (see  Q2,  Item  24). 

The  composite  results  described  above  and  in  previous 
sections  of  the  report  support  the  conclusion  that  AIRNET' s 
training  effectiveness  is  degraded  by  (a)  the  limited  range 
at  which  targets  can  be  detected  and  identified  with  unaided 
vision  and  (b)  the  limited  range  at  which  specific  targets 
can  be  identified  with  the  AIRNET  FLIR.  Target  detection  and 
IFF  identification  range  with  AIRNET  FLIR  appear  adequate  to 
support  effective  training. 


Collective  Task  Performance 

The  results  of  the  collective  task  performance 
assessment  are  more  difficult  to  interpret  than  the  results 
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of  the  assessment  of  performance  on  either  individual  flying 
tasks  or  target  detection  and  identification.  The  difficulty 
stems  primarily  from  inconsistencies  in  the  collective  task 
performance  ratings.  On  the  one  hand,  there  were  few 
collective  subtasks  for  which  performance  was  judged 
inadequate  by  a  large  proportion  of  crewmembers,  SMEs,  or 
both.  On  the  other  hand,  there  are  at  least  four  reasons  to 
believe  that  performance  on  collective  tasks  was,  in  fact, 
inadequate  for  more  of  the  tasks  than  was  indicated  by  the 
performance  ratings. 

First,  there  were  35  collective  subtasks  for  which 
performance  was  judged  inadequate  on  two  or  more  iterations 
(by  two  crews,  two  SMEs,  or  one  of  each) .  These  findings 
show  that  there  are  a  substantial  number  of  collective 
subtasks  for  which  performance  was  not  consistently  judged 
adequate.  Second,  performance  on  many  collective  subtasks 
was  judged  adequate  even  though  the  collective  subtasks 
require  crewmembers  to  perform  individual  flying  tasks  that 
clearly  cannot  be  performed  to  standards  in  AIRNET.  The  most 
conspicuous  examples  include  adequate  ratings  on  the 
performance  of  collective  subtasks  that  require  crewmembers 
to  perform  decelerations  and  target  detection  and 
identification.  Third,  the  SMEs  who  performed  the  post  hoc 
assessment  agreed  that  the  35  collective  subtasks  identified 
as  problem  subtasks  cannot  consistently  be  performed 
adequately  in  AIRNET;  moreover,  they  had  no  difficulty 
identifying  AIRNET  components  that  contributed  to  the 
performance  difficulty. 

Finally,  the  researchers  observed  inconsistencies 
between  the  collective  task  performance  ratings  and  the 
spontaneous  comments  of  crewmembers  and  SMEs.  After 
observing  the  collective  task  performance  and  hearing 
crewmember  complaints  about  the  difficulty  in  performing  some 
of  the  collective  tasks,  the  researchers  expected  far  more 
inadequate  ratings  than  were  found.  For  these  four  reasons, 
the  authors  have  concluded  that  the  rating  data  represent  an 
underestimate  of  the  frequency  with  which  collective  tasks 
were  performed  inadequately  in  AIRNET. 

An  examination  of  the  35  collective  subtasks  identified 
as  problem  subtasks  (see  Table  5)  and  the  results  of  the  SME 
post  hoc  assessment  suggest  that  problems  in  performing 
collective  subtasks  stem  mainly  from  difficulty  in 
controlling  the  simulated  aircraft  and  difficulty  in  seeing 
(unaided  and  FLIR)  what  must  be  seen  to  perform  certain 
collective  subtasks  effectively.  For  instance,  the  results 
of  the  post  hoc  assessment  showed  that  the  four  most  frequent 
contributors  to  inadequate  performance  of  collective  subtasks 
are  either  aircraft  control  problems  (control  during  terrain 
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flight  and  decelerations)  or  vision  problems  (limited  target 
detection/identification  range  and  limited  FOV) .  The  results 
also  indicate  that  performance  on  some  collective  subtasks  is 
adversely  affected  by  the  sparseness  of  the  topography  in  the 
visual  system's  topographic  data  base.  Excessively  sparse 
topography  degraded  performance  on  such  tasks  as  deployment 
to  cover,  masking,  employing  suppressive  fire,  firing  weapons 
at  maximum  ranges,  and  performing  SAR.  Table  6  and  the 
supporting  discussion  provide  more  information  about  the 
specific  subtasks  adversely  affected  by  each  of  the  AIRNET 
component  design  shortcomings. 

Because  an  important  objective  of  this  research  was  to 
identify  ways  in  which  AIRNET' s  training  effectiveness  may  be 
increased,  much  of  the  description  and  discussion  of  results 
necessarily  have  focused  on  actual  or  potential  shortcomings 
in  the  design  of  AIRNET  components.  However,  it  is  equally 
important  that  readers  have  a  clear  understanding  of  AIRNET' s 
strengths.  In  this  regard,  it  is  important  to  emphasize  that 
the  composite  data  revealed  few  instances  in  which  AIRNET 
lacked  the  capability  that  trainers  need  to  create  and 
implement  a  tactically  valid  attack  team  training  scenario. 
The  only  important  capabilities  that  participants  judged  to 
be  lacking  in  AIRNET  are  the  lack  of  a  remote  designation 
capability  and  the  lack  of  an  ATHS  capability. 

The  composite  results  of  the  collective  task  assessment 
support  the  following  conclusions. 

•  Crewmembers  encounter  problems  in  performing  a 
substantial  number  of  collective  subtasks.  The 
problems  stem  mainly  from  difficulty  in  controlling 
the  simulated  aircraft  and  difficulty  in  seeing 
(unaided  and  FLIR)  what  must  be  seen  to  perform 
collective  subtasks  effectively. 

•  The  specific  AIRNET  component  design  shortcomings  that 
degrade  performance  on  collective  tasks  are  limited 
target  acquisition  range,  limited  lateral  FOV, 
aircraft  control  difficulties  (especially  during 
terrain  flight,  deceleration,  and  weapons  firing),  and 
excessively  sparse  and/or  unrealistic  topographic 
features . 

•  With  few  exceptions,  AIRNET  has  the  fundamental 
capabilities  needed  to  develop  and  implement 
tactically  valid  training  scenarios  for  Army  aviation 
attack  teams.  Remote  target  designation  and  ATHS  are 
the  only  important  capabilities  that  are  lacking  in 
the  current  version  of  AIRNET. 
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Questionnaire  Results 


The  questionnaires  employed  in  this  research  were 
designed  to  assess  participants'  judgments  about  (a)  the 
adequacy  of  AIRNET  components  for  performing  specific  tasks, 
(b)  the  range  at  which  targets  can  be  acquired  in  AIRNET 
(unaided  and  FLIR) ,  and  (c)  the  probable  effectiveness  of 
AIRNET  for  training  specific  tasks .  Since  participants  were 
required  to  rate  the  adequacy  of  AIRNET  components  for 
performing  specific  tasks,  the  responses  provide  opinion  data 
on  both  AIRNET  component  adequacy  and  problem  tasks .  That 
is,  if  a  large  proportion  of  participants  judge  a  component 
to  be  inadequate  for  performing  a  specific  task,  it  can  be 
inferred  that  the  task  cannot  be  performed  adequately  and  the 
AIRNET  component  in  question  contributes  to  the  inadequate 
performance . 

The  following  discussion  reflects  the  authors'  best 
judgments  about  whether  the  crewmembers'  responses  to 
questionnaire  items  indicate  the  presence  of  an  AIRNET 
component  design  shortcoming  that  is  important  enough  to 
adversely  affect  crewmembers'  judgments  about  AIRNET' s 
training  value.  In  general,  the  authors  have  judged  that  a 
potentially  important  problem  exists  if  40%  or  more  of  the 
crewmembers  judged  an  AIRNET  component  to  be  inadequate  for 
performing  one  or  more  tasks .  It  is  recognized  that  some 
readers  may  use  different  criteria  for  defining  a  problem. 

For  this  reason,  readers  are  encouraged  to  examine  carefully 
the  questionnaire  results  presented  in  the  previous  section 
of  the  report  and  to  draw  their  own  inferences  about  the 
presence  and  severity  of  AIRNET  component  design  shortcomings 
and  the  extent  to  which  the  shortcomings  are  likely  to 
influence  trainees'  perceptions  of  AIRNET' s  training  value. 

Responses  to  questionnaire  items  that  address  AIRNET 
component  design  indicate  that  the  components  most  often 
rated  inadequate  are  the  visual  system  and  the  simulated 
aircraft's  flight  characteristics.  Taken  as  a  whole, 
crewmembers'  responses  about  the  adequacy  of  the  visual 
system  reflect  the  opinion  that  the  lateral  FOV  is  too  small 
and  that  the  computer  generated  visual  scene  is  not 
sufficiently  realistic.  The  data  show  that  crewmembers 
believe  that  the  limited  FOV  and  the  unrealistic  visual  scene 
degrade  performance  on  many,  but  by  no  means  all,  of  the 
tasks  investigated.  In  addition,  many  crewmembers  indicated 
that  the  visual  system  creates  eye  fatigue  (50%) ,  eye  strain 
(43%),  and  blurred  vision  (29%).  Although  these  results 
cannot  be  assumed  unimportant,  this  research  yielded  no  data 
with  which  to  judge  whether  the  frequency  and  severity  of 
these  visual  symptoms  are  great  enough  to  degrade  training 
effectiveness.  Finally,  the  questionnaire  data  indicate  that 
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a  substantial  proportion  of  the  crewmembers  believe  that  the 
visual  displays  are  positioned  too  high  and  too  far  from  the 
crewmember.  The  questionnaire  data  did  not  indicate  a 
problem  in  either  the  stability  or  smoothness  of  the  visual 
system. 

Responses  to  questionnaire  items  that  address  AIRNET' s 
flight  characteristics  clearly  indicate  that  crewmembers 
consider  AIRNET' s  equations  of  motion  to  be  inadequate  for 
performing  a  substantial  number  of  important  flying  task. 
These  results  are  entirely  consistent  with  the  results  of  the 
assessments  of  both  individual  and  collective  task 
performance.  The  questionnaire  results  also  indicate  that  a 
substantial  percentage  of  crewmembers  consider  AIRNET 
inadequate  in  its  responses  to  the  control  inputs  required  to 
perform  TADS  slewing,  weapons  selection,  30  mm  cannon  firing, 
and  doppler  entry. 

Other  AIRNET  components  or  design  attributes  judged 
inadequate  by  many  crewmembers  include  its  communication 
system,  the  positioning  of  the  cockpit  instrument  displays, 
and  the  comfort  of  the  seats.  Discussions  with  crewmembers 
indicated  that  the  communication  system  was  judged  inadequate 
mainly  because  the  controls  are  difficult  to  operate  and  the 
system  has  too  few  communication  channels.  Discussions  with 
crewmembers  failed  to  reveal  any  consistent  opinions  about 
the  manner  in  which  the  cockpit  instrument  displays  should  be 
repositioned. 

Crewmembers'  ratings  of  AIRNET' s  training  value  provide 
useful  information  about  the  relative  value  of  AIRNET  for 
training  different  tasks.  However,  as  was  stated  earlier, 
readers  are  cautioned  against  using  such  data  to  draw 
inferences  about  the  absolute  training  value  of  AIRNET.  The 
results  show  that  AIRNET' s  training  value  tends  to  be  rated 
highest  for  collective  tasks--the  tasks  that  AIRNET  was 
designed  to  train.  When  AIRNET' s  training  value  was  rated 
low  for  a  collective  task,  it  appears  that  the  low  ratings 
stem  from  shortcomings  in  one  or  more  of  AIRNET' s  components 
rather  than  from  a  lack  of  capability  to  simulate  tactically 
valid  training  scenarios. 

In  general,  the  questionnaire  data  support  all  of  the 
conclusions  drawn  from  the  assessment  of  individual  task 
performance,  target  detection  and  identification  performance, 
and  collective  task  performance.  The  questionnaire  data 
support  the  conclusion  that  Army  aviators'  judgments  about 
the  training  value  of  AIRNET  are  likely  to  be  adversely 
affected  by  what  they  perceive  to  be  shortcomings  in  the 
design  of  certain  AIRNET  component s . 


53 


The  results  of  this  research  provide  evidence  that 
shortcomings  in  the  design  of  certain  AIRNET  components  have 
an  adverse  effect  on  the  performance  of  many  of  the  tasks 
that  must  be  performed  to  accomplish  team  training  in  AIRNET. 
Moreover,  the  results  indicate  that  the  same  design 
shortcomings  may  have  an  adverse  effect  on  crewmembers' 
perceptions  of  AIRNET1 s  training  value.  The  results  do  not 
provide  the  data  with  which  to  estimate  the  extent  to  which 
AIRNET' s  training  value  is  degraded  by  these  shortcoming  or, 
conversely,  the  extent  t  >  which  AIRNET' s  training  value  would 
be  increased  by  the  many  alternative  modifications  that  are 
possible  for  improving  the  components  found  to  be  lacking. 
Only  transfer-of-training  research  can  provide  the  data 
needed  to  make  such  judgments.  Even  so,  the  results  of  this 
research  are  sufficiently  compelling  to  justify 
recommendations  to  evaluate  the  cost  and  feasibility  of 
alternative  modifications  that  may  eliminate  or  reduce  the 
AIRNET  component  design  shortcomings  found  to  adversely 
affect  performance,  crewmembers'  attitudes  about  training 
effectiveness,  or  both. 

The  recommendation  considered  to  have  the  highest 
priority  is  to  perform  analytical  research  to  identify 
alternative  methods  for  improving  the  flight  characteristics 
of  the  simulated  aircraft  and  estimate  the  cost  and  benefits 
of  promising  alternatives.  The  suitability  of  equations  of 
motion  developed  for  other  helicopter  simulators  should  be 
assessed  before  attempts  are  made  to  develop  new  ones.  An 
analytical  assessment  by  experts  in  helicopter  simulation 
should  be  adequate  to  identify  existing  equations  of  motion 
that  will  support  the  tasks  that  must  be  performed  in  AIRNET. 
If  so,  the  key  task  is  to  determine  whether  suitable 
equations  of  motion  can  be  implemented  on  AIRNET' s  existing 
computers  or  affordable  computers  with  enhanced  capabilities. 

The  recommendation  with  the  second  highest  priority  is 
to  identify  and  evaluate  alternative  methods  for  increasing 
the  range  at  which  targets  can  be  detected  and  identified 
with  unaided  vision  and  with  FLIR.  Although  the  target 
detection  range  with  the  AIRNET  FLIR  may  be  adequate  for 
training,  some  increase  in  training  realism  may  result  from 
increasing  the  range  by  1  or  2  km.  Increasing  the  visual 
system's  resolution  is  likely  to  be  an  excessively  costly  way 
to  increase  target  acquisition  range.  Hence,  it  appears 
worthwhile  to  consider  alternative  techniques  such  as 
increasing  the  absolute  size  of  targets  or  making  target  size 
variable  and  inversely  related  to  viewing  range.  In 
considering  such  alternatives,  it  should  be  kept  in  mind  that 
AIRNET  is  not  intended  to  teach  target  detection  and 


identification  skills.  Rather,  target  detection  and 
identification  in  AIRNET  should  be  viewed  as  a  prerequisite 
for  performing  effective  training  on  many  of  the  collective 
tasks . 

A  third  recommendation  is  to  identify  and  evaluate 
alternative  methods  for  increasing  the  realism  of  the  visual 
system's  scene  elements.  This  effort  should  begin  with  a 
compilation  of  additional  information  about  the  scene 
elements  that  contribute  to  inadequate  performance  of  key 
tasks.  The  present  results  provide  some  information  about 
scene  elements  that  were  judged  too  unrealistic,  but  the 
research  was  not  designed  to  provide  comprehensive 
information  about  the  adequacy  of  scene  element  realism. 

Once  the  requirements  for  increased  realism  are  better 
understood,  methods  can  be  identified  to  meet  these 
requirements  by  modifying  the  AIRNET  topographic  data  base 
and  the  existing  visual  system  and  replacing  the  existing 
visual  system  with  one  that  has  more  capability. 

A  fourth  recommendation  is  to  identify  and  evaluate 
alternative  methods  for  increasing  the  lateral  FOV  of  the 
visual  system.  This  effort  should  begin  by  obtaining 
additional  information  about  (a)  the  full  range  of  tasks  for 
which  performance  is  adversely  affected  by  a  limited  lateral 
FOV  and  (b)  the  amount  that  the  FOV  would  need  to  be 
increased  to  support  adequate  performance  of  these  tasks. 
Since  increasing  the  actual  FOV  of  a  visual  system  is  certain 
to  be  costly,  methods  should  be  considered  for  increasing  the 
funct ional  FOV  without  increasing  the  actual  FOV.  For 
instance,  it  may  be  possible  to  use  auditory  or  symbolic 
information  to  advise  aviators  of  the  presence  or  location  of 
important  objects  that  are  visible  from  an  aircraft  cockpit 
but  are  located  beyond  the  limits  of  AIRNET' s  visual  system. 

Finally,  it  is  recommended  that  methods  be  identified 
and  evaluated  for  (a)  improving  the  communication  system's 
controls  and  increasing  the  number  of  communication  channels; 
(b)  improving  the  positioning  of  the  visual  system  displays 
and  the  cockpit  instrument  panel;  (c)  improving  AIRNET' s 
response  to  control  inputs  for  TADS  slewing,  weapon 
selection,  30  mm  cannon  firing,  and  doppler  entry;  and  (d) 
improving  the  comfort  of  the  seats  in  the  operator  stations. 
Although  it  seems  unlikely  that  such  improvements  would 
result  in  major  improvements  in  AIRNET' s  training 
effectiveness,  they  may  hav^  a  significant  effect  on  AIRNET' s 
user  acceptance. 
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APPENDIX  A 


LISTING  OF  COLLECTIVE  TASKS  AND  SUBTASKS 
INVESTIGATED  FOR  SCOUT  CREWMEMBERS  AND 
ATTACK  CREWMEMBERS 


A-l 


SCOUT  HELICOPTER  TASKS /SUBTASKS 

Conduct  Movement  to  a  Holding  Area 

•  Move  to  a  holding  area 

•  Conduct  passage  of  lines 

•  Unit  arrives  at  the  holding  area 

Use  Passive  Air  Defense  Measures 

•  Company  utilizes  passive  air  defense  measures  while  en 
route 

Conduct  Tactical  Air  Movement  as  Part  of._a  Movement  to 

Contact  or  an  Air  Assault  Security 

•  Company  conducts  movement 

•  Conduct  passage  of  lines 

Move  to  and  Occupy,  a  Battle  Position 

•  Reconnoiter  the  battle  position 

•  Occupy  the  battle  position 

•  Perform  actions  on  contact 

•  Depart  the  battle  position 

Use  Countermeasures  Against  Enemy  Air  Defense  Artillery 
(ADA)  to  Ensure  Aircraft  Survivability 

•  Aircrews  use  terrain  flight  techniques  and  passive  ADA 
countermeasures  in  accordance  with  the  factors  of 
mission,  enemy,  terrain,  troops,  and  time  available 
(METT-T) 

Report  Intelligence , Data 

•  Scout /weapons  team  leader  or  platoon  leaders  submit 
spot  report 

•  Scout/weapons  team  leader  or  platoon  leaders  report 
bombing,  shelling  and  mortar,  rocket  and  aircraft  fire 

•  Scout/weapons  team  leader  or  platoon  leaders  submit 
battle  damage  assessment  reports 

Establish  Contact 

•  Lead  elements  react  to  initial  contact  with  opposing 
forces  (OPFOR) 

•  Perform  actions  on  contact 


A-2 


•  Conduct  target  handovers 

•  Employ  close  air  support 

•  Call  for  and  adjust  artillery  fire 


•  Company  commander  issues  the  operations  order, 

designates  air  combat  teams,  reviews  aircrew  standing 
operating  procedures  (SOP) 


•  Company  conducts  forward  arming  and  refueling  point 
(FARP)  operations 

•  Company  commander  orders  the  company  to  depart  the 
rally  point 

•  Company  reorganizes  at  the  forward  assembly  area 


•  Execute  new  mission  guidance 


[>[•)  <!•  i*  wtw  Wj  »iti 


Unit  conducts  JAAT  operation 


ATTACK  HELICOPTER  TASKS /SUBTASKS 


Conduct  Movement  to  Holding  Area 

•  Move  to  the  holding  area 

•  Conduct  passage  of  lines 

•  Unit  arrives  at  the  holding  area 

Use  Passive  Air  Defense  Measures 

•  Company  utilizes  passive  air  defense  measures  while  en 
route 

Move  to  .and  Occupy  a  Battle  Position 

•  Occupy  battle  position 

Use  Countermeasures.  Against  Enemy  to  Ensure  ADA  Aircraft 
Survivability 

•  Aircrews  use  terrain  flight  techniques  and  passive  ADA 
countermeasures  in  accordance  with  the  factors  of  METT-T 

Report  Intelligence  Data 

•  Scout/weapons  team  leader  or  platoon  leaders  submit 
spot  report 

•  Scout/weapons  team  leader  or  platoon  leaders  report 
bombing,  shelling  and  mortar,  rocket  and  aircraft  fire 

•  Scout /weapons  team  leader  or  platoon  leaders  submit 
battle  damage  assessment  reports 

Establish  Contact 

•  Lead  elements  react  to  initial  contact  with  OPFOR 
Engage  Targets 

•  Attack  platoons  engage  targets 
Conduct  Deliberate  Air  Attack 

Conduct  Hasty  Air  Attack 

•  Company  commander  issues  the  operations  order, 
designates  air  combat  teams,  reviews  aircrew  SOP 

Return  to  Assembly  Area  and  Prepare  for  Future  Operations 

•  Company  conducts  FARP  operations 

•  Company  commander  orders  the  company  to  depart  the  rally 
point 

•  Company  reorganizes  at  the  forward  assembly  area 
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Depart  the  battle  position 
Execute  new  mission  guidance 


Conduct  JAAT  Operations 

•  Company  commander  coordinates  close  air  support  assets 

•  Company  commander  and  platoon  leaders  conduct  fire 
support  coordination 

•  Unit  conducts  JAAT  operations 


APPENDIX  B 

SPECIMENS  OF  RATING  FORMS  USED  TO 
ASSESS  COLLECTIVE  TASKS 
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ENGAGE  TARGETS  (TASK  01-2-0105) 
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COMMENTS : 


USE  COUNTERMEASURES  AGAINST  ENEMY  ADA  TO  ENSURE  AIRCRAFT  SURVIVABILITY  (TASK  01-2-0301) 

AIRCREWS  USE  TERRAIN  FLIGHT  TECHNIQUES  AND  PASSIVE  ADA  COUNTERMEASURES  IAW 
THE  FACTORS  OF  METT-T 
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COMMENTS : 


ENGAGE  TARGETS  (TASK  01-2-0105) 
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APPENDIX  C 
CREWMEMBER  QUESTIONNAIRE  (Ql) 
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Date 

Name 

Aircraft : 

Crewseat : 

Scout 

Pilot 

Unit 

Attack 

Copilot 

AIRNET  RATING  SCALE 
Technical  Performance  Issues 


Use  the  5-point  scale  shown  below  to  rate  the  technical 
performance  of  the  AIRNET  device .  Circle  the  performance 
rating  for  each  of  the  areas  listed.  Use  the  back  of  this 
page  for  additional  comments  if  more  space  is  required. 


RATING  SCALE 


Totally 

inadequate 


Somewhat  Just  adequate 
inadequate  enough 


More  than 
adequate 


Much  more 
than  adequate 


j _ 2 _ 2 _ a _ a 


Visual  Fidelity 

OUT  THE  WINDOW  FIELD  OF  VIEW: 

1.  Is  the  field  of  view  wide  enough  to  enable  you  to  perform 
your  mission? 

5 
5 
5 
5 
5 
5 
5 
5 
5 
5 
5 
5 
5 

Comments:  _ 


Forward  flight  0  to  50  knots  1234 
Forward  flight  50  to  100  knots  1234 
Forward  flight  greater  than  100  knots  1234 
Low  level  flight  1234 
Contour  flight  1234 
NOE  flight  1234 
Hovering  1234 
Deceleration  1234 
Normal  take-off /landing  1234 
Threat  detection/avoidance  1234 
Target  acquisition  1234 
Situation  awareness  1234 
Formation  flight  1234 
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RATING  SCALE 


Totally  Somewhat  Just  adequate  More  than  Much  more 
inadequate  inadequate  enough  adequate  than  adequate 


1 _ 2 _ 2 _ 4 _ 5. 


2 .  Does  the  forward  visual  scene  contain  sufficient  visual 
cues  to  perform  the  mission? 

5 
5 
5 
5 
5 
5 
5 
5 
5 

Comments : 


Forward  flight  0  to  50  knots  1234 
Forward  flight  50  to  100  knots  1234 
Forward  flight  greater  than  100  knots  1234 
Low  level  flight  1234 
Contour  flight  1234 
NOE  flight  1234 
Hovering  1234 
Deceleration  1234 
Normal  take-off /landing  1234 


3.  Is  the  stability  of  the  displayed  visual  scene  acceptable? 


At  a  hover 
0  to  50  knots 
50  to  100  knots 
greater  than  100  knots 


1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 


Comments : 


4.  Are  the  topographical  features  sufficiently  discernible 
to  support  flight  tasks? 


Low  level  flight  0  to  50  knots 
Low  level  flight  50  to  100  knots 
Contour  flight  0  to  50  knots 
Contour  flight  50  to  100  knots 
NOE  flight  0  to  50  knots 
NOE  flight  50  to  100  knots 
Target  acquisition 
Situation  awareness 


1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 


Comments : 


RATING  SCALE 


Totally  Somewhat  Just  adequate  More  than  Much  more 
inadequate  inadequate  enough  adequate  than  adequate 


5.  Is  the  displayed  visual  scene  adequate  to  support  gunnery 
tasks  in  various  flight  regimes? 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

Comments : 


Low  level  flight  0  to  50  knots 
Low  level  flight  50  to  100  knots 
Contour  flight  0  to  50  knots 
Contour  flight  50  to  100  knots 
NOE  flight  0  to  50  knots 
NOE  flight  50  to  100  knots 
Target  acquisition 
Situation  awareness 


6.  Check  the  range  at  which  you  feel  the  AIRNET  visual 
system  adequately  supports  target  acquisition  tasks. 

_  Not  at  all 

_  200  to  500  meters 

_  500  to  1000  meters 

_  1000  to  1500  meters 

_  1500  to  2000  meters 

_  2000  to  2500  meters 

_  2500  to  3000  meters 

_  3000  to  3500  meters 

_  More  than  3500  meters 

Comments  _ _ 
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RATING  SCALE 


Totally  Somewhat  Just  adequate  More  than  Much  more 
inadequate  inadequate  enough  adequate  than  adequate 

1 _ 2 _ 2 _ 4 _ 5. 


VISUAL  REPRESENTATION  FROM  THE  FLIR : 

7.  Check  the  range  at  which  you  feel  the  AIRNET  FLIR 

adequately  supports  sensor  based  target  acquisition 
tasks . 

_  Not  at  all 

_  0  to  1000  meters 

_  1000  to  2000  meters 

_  2000  to  3000  meters 

_  3000  to  4000  meters 

_  4000  to  5000  meters 

_  5000  to  6000  meters 

_  6000  to  7000  meters 

_  More  than  7000  meters 

Comments : 


FLIGHT  CHARACTERISTICS: 

8.  Does  the  device  fly  enough  like  a  helicopter  to  allow  the 
crewmembers  to  perform  their  mission? 


1  2  3  4  5 


Comments : 


9.  Are  the  demands  on  pilot  attention  required  to  fly  the 
AIRNET  simulated  aircraft  comparable  to  the  real 
aircraft? 


1  2  3  4  5 


Comments : 
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RATING  SCALE 


Totally  Somewhat  Just  adequate  More  than  Much  more 
inadequate  inadequate  enough  adequate  than  adequate 


10.  Are  the  handling  characteristics  associated  with  use  of 
the  flight  controls  reasonable  comparable  to  the 
aircraft? 

When  input  is  made  to  the  cyclic  12345 

When  input  is  made  to  the  pedals  12345 

When  input  is  made  to  the  collective  12345 

Comments 


COMMUNICATION  SYSTEM: 

11.  Does  the  communication  system  allow  crewmembers  to 
transmit  information  successfully  to: 

One  another  12345 

The  tactical  operations  center  (TOC) 

commander  12345 

Other  aircraft  12345 

Comments : 


12.  Does  the  communication  system  allow  crewmembers  to 
receive  information  successfully  from: 

One  another 
The  TOC  commander 
Other  aircraft 


Comments : 


1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 


13.  Does  the  communication  system  allow  crewmembers  to 
monitor  two  (2)  channels  simultaneously? 

_  Not  at  all 

_  Totally  inadequately 

_  Somewhat  inadequately 

_  Just  adequate  enough 

_  More  than  adequate 

_  Much  more  than  adequate 


Comments : 


APPENDIX  D 
CREWMEMBER  QUESTIONNAIRE  (Q2) 
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Date _ Name _ 

Aircraft:  _  Scout  Crewseat : _  Pilot 

_  Attack  _  Copilot 


AIRNET  RATING  SCALE 

Use  the  scale  shown  below  to  rate  the  performance  of  the 
AIRNET  device.  Circle  the  performance  rating  for  each  of  the 
areas  listed.  Use  the  back  of  page  for  additional  comments 
if  extra  space  is  required.  Skip  questions  that  do  not  apply 
to  your  crewseat  or  aircraft. 


RATING  SCALE 


Totally 

inadequate 


Somewhat  Just  adequate 
inadequate  enough 


More  than 
adequate 


Much  more 
than  adequate 


1 _ 2 _ 3 _ 4 _ a 


Technical  Performance 


1.  Does  the  AIRNET  local  area  network  provide  real  time 
interactive  simulation,  with  no  apparent  delay,  when 
inputs  are  made  to  the  following  systems: 


a) 

Collective 

1 

2 

3 

4 

5 

b) 

Cyclic 

1 

2 

3 

4 

5 

c) 

Pedals 

1 

2 

3 

4 

5 

d) 

Hover  hold 

1 

2 

3 

4 

5 

e) 

Weapon  selection 

1 

2 

3 

4 

5 

f) 

FLIR/TV  selection 

1 

2 

3 

4 

5 

g) 

Slewing  of  the  TADS 

1 

2 

3 

4 

5 

h) 

Doppler  entry 

1 

2 

3 

4 

5 

i) 

Doppler  correction 

1 

2 

3 

4 

5 

j) 

Firing  30  mm  cannon 

1 

2 

3 

4 

5 

k) 

Firing  HELLFIRE 

1 

2 

3 

4 

5 

1) 

Firing  Stinger 

1 

2 

3 

4 

5 

m) 

Comments 

Lasing  targets 

1 

2 

3 

4 

5 
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Visual  Representation 


Is 

to 

the  field  of  view  wide 
perform  their  mission? 

enough  to  enable 

crewmembers 

a) 

Forward  flight 

1 

2 

3 

4 

5 

b) 

NOE  flight 

1 

2 

3 

4 

5 

c) 

NOE  deceleration 

1 

2 

3 

4 

5 

d) 

Hovering 

1 

2 

3 

4 

5 

e) 

Deceleration 

1 

2 

3 

4 

5 

f) 

Threat  detection 

1 

2 

3 

4 

5 

g) 

Threat  avoidance 

1 

2 

3 

4 

5 

h) 

Formation  flight 

1 

2 

3 

4 

5 

i) 

Situation  awareness 

1 

2 

3 

4 

5 

Comments : 


3.  Is  the  forward  visual  scene  realistic  enough? 


a)  For  navigation 

b)  Calling  fires 

c)  Adjusting  fires 

d)  Altitude  detection 


1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 


Comments : 


Is 

the  visual  scene  smooth  and  continuous 

at 

all 

times? 

a) 

During  flight  0  to  50  knots 

1 

2 

3 

4 

5 

b) 

During  flight  50  to  100  knots 

1 

2 

3 

4 

5 

c) 

During  flight  above  100  knots 

1 

2 

3 

4 

5 

d) 

When  encountering  the  semi- 
automated  forces 

1 

2 

3 

4 

5 

Comments : 
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5.  Are  the  topographical  features  realistic  enough  for: 

4  5 

4  5 

4  5 

4  5 

4  5 

Comments : 


a)  Negotiating  flight 

b)  Navigation 

c)  Masking/Unmasking 

d)  NOE  flight 

e)  Altitude  detection 


1 

1 

1 

1 

1 


2 

2 

2 

2 

2 


3 

3 

3 

3 

3 


6.  Does  the  graphic  representation  allow  for  target 
detection  (both  ground  and  air  vehicles) : 


Out  the  Window 

_  Not  at  all 

_  200  to  500  m 

_  500  to  1000  m 

_  1000  to  1500  m 

_  1500  to  2000  m 

_  2000  to  2500  m 

_  2500  to  3000  m 

_  3000  to  3500  m 

_  greater  than  3500  m 


FLIR/TV  View 

_  Not  at  all 

_  0  to  1000  m 

_  1000  to  2000  m 

_  2000  to  3000  m 

_  3000  to  4000  m 

_  4000  to  5500  m 

_  5000  to  6000  m 

_  6000  to  7000  m 

_  greater  than  7000  m 


Comments 


Hardware  Fidelity 


7.  Does  the  AIRNET  sound  system  accurately  portray  all 
systems  and  weapon  sounds? 


a)  Rotor  system 

b)  30  mm  gun 

c)  HELLFIRE 

d)  AT AS 

e)  ATAS  tone 

f)  Own  vehicle  being  hit 

g)  Artillery  impact 

h)  APR  39  tones 


1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 


Comments : 
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8.  Does  the  gunner's  station  allow  the  gunner  to 

realistically  perform  those  tasks  that  he  would  perform 
in  the  attack  helicopter? 

a)  Weapons  selection 

b)  Search  for  targets  through  FLIR 

c)  Radio  communication 

d)  Similar  workload  levels 

e)  Necessary  lights,  switches, 

and  gauges 

Comments 


9.  Does  the  copilot's  station  allow  the  copilot  to 

realistically  perform  those  tasks  that  he  would  perform 
in  the  scout  helicopter? 

a)  Navigation 

b)  Target  identification 

c)  Radio  communication 

d)  Similar  workload  levels 

Comments 


10.  Does  the  pilot's  station  allow  the  pilot  to 

realistically  perform  those  tasks  that  he  would  perform 
in  the  scout  helicopter? 

a)  Negotiate  terrain 

b)  Seek  positions  to  mask 

c)  Anticipate  and  negotiate  obstacles 

d)  Fly  the  aircraft  in  any  intended 

direction 

Comments 


11.  Are  all  switches  and  gauges  necessary  for  the  pilot  to 

perform  the  mission  present  in  the  simulator?  YES  NO 

Explain _ 


1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 

1  2  3  4  5 


1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 


1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 

1  2  3  4  5 
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12.  Are  all  switches  and  gauges,  etc.  in  a  similar  location  and 
configuration  to  those  present  in  the  aircraft?  YES  NO 

Explain _ 


13.  The  AIRNET  attack  module  allows  the  pilot  and  copilot  to 
maintain  visibility  with  one  another.  Does  this 
arrangement 

[  ]  Increase/improve  teamwork 

[  ]  Increase/improve  communication  between  pilot  and 
copilot 

[  ]  Not  affect  performance  in  any  way 

[  ]  Decrease  the  need  to  complete  communication 

between  the  pilot  and  copilot 

[  ]  Impair  performance  between  the  pilot  and  copilot 

Comments 


14.  The  arrangement  of  the  pilot  and  copilot  seat  in  the 
same  crew  module  is 

[  ]  Preferred 

[  ]  Makes  no  difference 

[  ]  Not  preferred 

Comments 


Training  and  Logistics 

15.  Do  the  AIRNET  exercises  demand  the  same  logistics 
support  requirements  as  the  real  aircraft? 

a)  Monitoring  ammunition  and  fuel  status  12345 

b)  Dispatch,  on  a  real  time  basis, 

of  fuel  and  ammunition  resupply  12345 


Comments 
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Communication 


16.  Use  the  following  scale  to  rate  the  amount  of  crew 

communication  that  is  required  to  perform  the  missions 
listed  below. 


No  Little  Moderate  High  Constant 

Commo  Commo  Commo  Commo  Commo 

_ 1 _ 2 _ 2 _ 4 _ 5 _ 


a) 

Contour  flight 

1 

2 

3 

4 

5 

b) 

NOE  flight 

1 

2 

3 

4 

5 

c) 

Target  acquisition 

1 

2 

3 

4 

5 

d) 

Target  engagement  - 

HELLFIRE 

1 

2 

3 

4 

5 

e) 

Target  engagement  - 

ATAS 

1 

2 

3 

4 

5 

f) 

Target  engagement  - 

30  mm 

1 

2 

3 

4 

5 

Comment 

s 

17 .  How  often  did  you  have  to  verbally  cross  check  your 

crewmate  to  ensure  he  had  completed  a  prescribed  task 
before  you  proceeded  with  your  tasks? 

[  ]  Almost  never 

[  ]  Infrequently 

[  ]  Occasionally 

[  ]  Frequently 

[  ]  Almost  always 

Comments 


18.  How  often  did  you  have  to  contact  the  other  aircraft  in 
order  to  complete  the  mission? 

[  ]  Almost  never 

[  ]  Infrequently 

[  ]  Occasionally 

[  ]  Frequently 

[  ]  Almost  always 

Comments  _  _ 
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Human  Factors 


Adequate 

enough 

3 


Much  more 
adequate 
_ 5 _ 


Totally  Somewhat 
inadequate  inadequate 

_ 1 _ 2 . . 


More  than 
adequate 
4 _ 


19.  Is  the  AIRNET  system  similar  to  the  actual  aircraft  for 
the  crewmembers  in  terms  of  comfort? 


a)  Seats  (for  at  least  2  hours) 

b)  Seats  (for  more  than  2  hours) 

c)  Location  of  collective 

d)  Distance  of  screens  from  the  pilot 

e)  Distance  of  screens  from 

Lhe  instrument  display 

f)  Location  of  the  instrument  display 

g)  Height  of  the  screens  with  respect 

to  the  pilot 

h)  Orientation  of  the  pedals 
Comments 


1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 


20.  Does  the  fidelity  of  the  visual  representation  differ 
from  the  actual  aircraft  in  terms  of 


a)  Eye  fatigue 

b)  Eye  strain 

c)  Blurred  vision 


1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 


Comments 


21.  Are  the  instrument  displays  easily  readable  and 

understandable?  YES  NO 

Comments 
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22 .  Does  use  of  the  AIRNET  device  result  in  any  adverse 
physiological  effects? 


Before  During  After 


a)  General  discomfort 

b)  Fatigue 

c)  Drowsiness 

d)  Headache 

e)  Difficulty  focusing 

f)  Sweating 

g)  Nausea 

h)  Difficulty  concentrating 

i)  Blurred  vision 

j)  Dizziness  (eyes  open) 

k)  Dizziness  (eyes  closed) 

l)  Vertigo 

m)  Faintness 

n)  Stomach  awareness 


Comments 


Negative  Habit  Transfer 

23.  Please  use  the  scale  from  the  Human  Factors  section  to 
respond  to  the  following  questions.  Due  to  your 
experience  in  the  AIRNET  device,  have  you  experienced 
any  adverse  performance  effects  in  the  actual  aircraft 
for  the  following  tasks? 


a) 

Forward  flight  0  to  50  knots 

1 

2 

3 

4 

5 

b) 

Forward  flight  50  to  100  knots 

1 

2 

3 

4 

5 

c) 

Forward  flight  greater  than  100  knots 

1 

2 

3 

4 

5 

d) 

Low  level  flight 

1 

2 

3 

4 

5 

e) 

Contour  flight 

1 

2 

3 

4 

5 

f) 

NOE  flight 

1 

2 

3 

4 

5 

g) 

Formation  flight 

1 

2 

3 

4 

5 

h) 

Hovering 

1 

2 

3 

4 

5 

i) 

Deceleration 

1 

2 

3 

4 

5 

j) 

Acceleration 

1 

2 

3 

4 

5 

k) 

Normal  takeoff /landing 

1 

2 

3 

4 

5 

3) 

Cyclic  overcontrol 

1 

2 

3 

4 

5 

m) 

Collective  input 

1 

2 

3 

4 

5 

n) 

Staying  in  trim 

1 

2 

3 

4 

5 

o) 

Communication  procedure 

1 

2 

3 

4 

5 

P) 

Increased  communication 

1 

2 

3 

4 

5 

q) 

Comments 

Decreased  communication 

1 

2 

3 

4 

5 
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Training  Value 


24.  Please  use  the  following  scale  to  determine  the  value  of 
the  AIRNET  device  as  it  is  currently  configured  for 
training  the  following  tasks. 

Not  Desirable  Very  Highly  Extremely 

desirable  desirable  desirable  desirable 

_ 1 _ 2 _ 2 _ 4 _ 5 _ 


a) 

Tactics/mission  planning 
and  briefing 

1 

2 

3 

4 

5 

b) 

Medium/high  altitude  tactics 

1 

2 

3 

4 

5 

c) 

Low  level  flight 

1 

2 

3 

4 

5 

d) 

Low  altitude  tactics 

1 

2 

3 

4 

5 

e) 

NOE  flight 

1 

2 

3 

4 

5 

f) 

Low  threat  tactics 

1 

2 

3 

4 

5 

g) 

High  threat  tactics 

1 

2 

3 

4 

5 

h) 

Tactical  formation 

1 

2 

3 

4 

5 

i) 

2-ship  tactics 

1 

2 

3 

4 

5 

j) 

4-ship  tactics 

1 

2 

3 

4 

5 

k) 

Visual  ID 

1 

2 

3 

4 

5 

1) 

Tactical  intercept 

1 

2 

3 

4 

5 

m) 

Missile  employment 

1 

2 

3 

4 

5 

n) 

Gun  employment 

1 

2 

3 

4 

5 

o) 

Safe  passage 

1 

2 

3 

4 

5 

P) 

Egress  tactics 

1 

2 

3 

4 

5 

q) 

Movement  to  contact 

1 

2 

3 

4 

5 

r) 

Communication  procedure 

1 

2 

3 

4 

5 

s) 

Call  for  fire 

1 

2 

3 

4 

5 

t) 

Adjust  fire 

1 

2 

3 

4 

5 

u) 

Close  air  support 

1 

2 

3 

4 

5 

v) 

Target  handover 

1 

2 

3 

4 

5 

w) 

Reconnaissance  missions 

1 

2 

3 

4 

5 

x) 

Screening  operations 

1 

2 

3 

4 

5 

y) 

Target  engagement 

1 

2 

3 

4 

5 

Comment, 

s 
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